NAVAL 

POSTGRADUATE 

SCHOOL 

MONTEREY,  CALIFORNIA 

THESIS 


A  COMPARISON  OF  CURRENT  NAVAL 
MARKSMANSHIP  TRAINING  VS.  SIMULATION-BASED 
MARKSMANSHIP  TRAINING  WITH  THE  USE  OF 
INDOOR  SIMULATED  MARKSMANSHIP  TRAINER 

(ISMT) 

by 

Tommy  J.  Getty 
March  2014 

Thesis  Advisor:  Quinn  Kennedy 

Second  Reader:  Joseph  Sullivan 


This  thesis  was  performed  at  the  MOVES  Institute 


Approved  for  public  release;  distribution  is  unlimited 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


REPORT  DOCUMENTATION  PAGE 


FonnApprovedOMBNa^0704-018^^ 
Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instruction, 
searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send 
comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for  reducing  this  burden,  to 
Washington  headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA 
22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188)  Washington  DC  20503. 

I.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

_ March  2014 _ Master’s  Thesis _ 

4.  TITLE  AND  SUBTITLE  A  COMPARISON  OF  CURRENT  NAVAL  5.  FUNDING  NUMBERS 

MARKSMANSHIP  TRAINING  VS.  SIMULATION-BASED  MARKSMANSHIP 
TRAINING  WITH  THE  USE  OF  INDOOR  SIMULATED  MARKSMANSHIP 
TRAINER  (ISMT) _ 

6.  AUTHOR(S)  Tommy  J.  Getty _ 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES)  8.  PERFORMING  ORGANIZATION 

Naval  Postgraduate  School  REPORT  NUMBER 

9.  SPONSORING  /MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES)  10.  SPONSORING/MONITORING 
N/A  AGENCY  REPORT  NUMBER 

II.  SUPPLEMENTARY  NOTES  The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official  policy 

or  position  of  the  Department  of  Defense  or  the  U.S.  Government.  IRB  protocol  number _ ^NPS.201  L0083-IR-EP7-A _ . 

12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  is  unlimited _ A _ 

13.  ABSTRACT  (maximum  200  words) 

The  Navy  small  arms  training  and  qualification  instruction  focused  on  dry  fire  and  simulators  should  be  utilized  when 
available.  However,  naval  personnel  often  do  not  have  access  to  dry  fire  training  opportunities  and  therefore  may  be 
at  risk  for  losing  perishable  marksmanship  skills.  The  purpose  of  the  study  was  to  determine  if  the  use  of  a  simulator 
is  at  least  as  effective  in  marksmanship  training  as  traditional  dry  fire  techniques. 

A  between-groups  study  with  a  random  selection  of  volunteers  (blocked  by  previous  marksmanship  experience) 
was  conducted  to  determine  if  participants  who  completed  simulation-based  training  showed  greater  improvement 
and  retention  of  marksmanship  skills  than  participants  completing  dry  fire  training.  The  main  measures  were  the 
qualification  scores,  average  shot  group  size,  shot  group  size  and  mean  point  of  impact  to  center  zeroing  point  length. 
The  experiment  utilized  a  simulated  M9  Berretta  for  the  qualification. 

Results  demonstrate  that  simulation  training  improved  performance  on  the  seven-yard  line  to  a  greater  extent  than 
current  naval  training.  Exploratory  analyses  suggest  that  simulation  training  may  be  most  beneficial  for  less- 
experienced  shooters,  and  that  a  minimum  of  two  weeks’  time  is  needed  to  detect  group  differences  in  the  retention  of 
skills. 


14.  SUBJECT  TERMS  Indoor  Simulated  Marksmanship  Trainer  (ISMT),  simulation  based  15.  NUMBER  OF 

marksmanship  training,  eurrent  naval  marksmanship  training,  standard  Navy  handgun  qualifieation  PAGES 


and  part-task  trainer. 

185 

16.  PRICE  CODE 

17.  SECURITY 

18.  SECURITY 

19.  SECURITY 

20.  LIMITATION  OF 

CLASSIFICATION  OF 

CLASSIFICATION  OF  THIS 

CLASSIFICATION  OF 

ABSTRACT 

REPORT 

PAGE 

ABSTRACT 

Unelassified 

Unelassified 

Unelassified 

uu 

NSN  7540-01-280-5500 

Standard  Form  298  (Rev.  2-89) 

Prescribed  by  ANSI  Std.  239-18 

1 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


n 


Approved  for  public  release;  distribution  is  unlimited 


A  COMPARISON  OF  CURRENT  NAVAL  MARKSMANSHIP  TRAINING  VS. 
SIMULATION  BASED  MARKSMANSHIP  TRAINING  WITH  THE  USE  OF 
INDOOR  SIMULATED  MARKSMANSHIP  TRAINER  (ISMT) 


Tommy  J.  Getty 
Lieutenant,  United  States  Navy 
B.S.,  Hampton  University,  2004 


Submitted  in  partial  fulfdlment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN  MODELING,  VIRTUAL  ENVIRONMENTS, 

AND  SIMULATION  (MOVES) 


from  the 


NAVAL  POSTGRADUATE  SCHOOL 
March  2014 


Author:  Tommy  J.  Getty 


Approved  by:  Quinn  Kennedy 

Thesis  Advisor 


Joseph  Sullivan 
Second  Reader 


Peter  Denning 

Chair,  Department  of  Computer  Science 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


IV 


ABSTRACT 


The  Navy  small  arms  training  and  qualification  instruction  focused  on  dry  fire  and 
simulators  should  be  utilized  when  available.  However,  naval  personnel  often  do  not  have 
access  to  dry  fire  training  opportunities  and  therefore  may  be  at  risk  for  losing  perishable 
marksmanship  skills.  The  purpose  of  the  study  was  to  determine  if  the  use  of  a  simulator 
is  at  least  as  effective  in  marksmanship  training  as  traditional  dry  fire  techniques. 

A  between-groups  study  with  a  random  selection  of  volunteers  (blocked  by 
previous  marksmanship  experience)  was  conducted  to  determine  if  participants  who 
completed  simulation-based  training  showed  greater  improvement  and  retention  of 
marksmanship  skills  than  participants  completing  dry  fire  training.  The  main  measures 
were  the  qualification  scores,  average  shot  group  size,  shot  group  size  and  mean  point  of 
impact  to  center  zeroing  point  length.  The  experiment  utilized  a  simulated  M9  Berretta 
for  the  qualification. 

Results  demonstrate  that  simulation  training  improved  performance  on  the  seven- 
yard  line  to  a  greater  extent  than  current  naval  training.  Exploratory  analyses  suggest  that 
simulation  training  may  be  most  beneficial  for  less-experienced  shooters,  and  that  a 
minimum  of  two  weeks’  time  is  needed  to  detect  group  differences  in  the  retention  of 
skills. 
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I. 


INTRODUCTION 


A.  STUDY  PURPOSE 

The  purpose  of  this  thesis  was  to  determine  whether  simulation-based 
marksmanship  training  with  the  Indoor  Simulated  Marksmanship  Trainer  (ISMT) 
improves  marksmanship  performance  to  a  greater  extent  than  current  naval 
marksmanship  training.  This  research  was  intended  to  provide  information  on  the 
performance  of  a  shooter  to  meet  a  qualification  standard  and  on  the  retention  of  skills 
over  time  if  basic  marksmanship  training  was  conducted  with  the  use  of  a  simulator.  Any 
benefits  on  different  methods  of  training  and  the  quality  of  the  feedback  to  a  trainee  can 
preserve  a  perishable  skill  like  marksmanship.  Naval  commands  could  use  the 
information  to  effectively  maintain  gun  qualifications  for  inport  duty  section  watch  bills 
and  constant  anti-terrorism  readiness. 

The  motivation  for  this  thesis  was  due  to  the  author's  personal  experiences  of  the 
lack  of  hands-on  weapons  training  prior  to  a  weapon  qualification.  The  majority  of  the 
naval  watch  standers  only  handle  a  gun  when  they  check  in  and  out  of  the  armory.  Sailors 
may  know  what  needs  to  happen  during  force  protection  exercises  but  may  not  have  the 
marksmanship  skills  required  to  effectively  employ  their  weapon  for  deadly  force.  The 
vessel,  board,  search  and  seizure  (VBSS)  team  endured  an  extensive  training  program  but 
the  majority  of  ship’s  crew  does  not  receive  in-depth  marksmanship  training  or 
experience  with  live  fire.  In  many  cases,  when  a  sailor  failed  to  qualify  on  the  standard 
navy  handgun  qualification  course  of  fire  (COF)  they  simply  were  ordered  to  the  back  of 
the  line  to  shoot  again.  The  anti-terrorism  watch  officer,  chief  of  the  guard,  and  topside 
rovers  are  a  few  watch  standers  that  rely  heavily  on  the  sailors’  ability  to  remember  pre¬ 
planned  responses.  There  are  pre-requisite  weapons  qualification  required  to  stand  those 
watches  and  the  quality  of  that  training  is  critical.  Sailors  are  required  to  stay  sharp  and 
have  a  high  level  of  knowledge  for  pre-planned  responses  to  know  when  to  use  deadly 
force,  but  it  is  equally  important  to  learn,  acquire  and  practice  marksmanship  skills  for 
inherent  right  of  self-defense  of  any  naval  unit. 
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This  study  attempted  to  identify  any  benefits  in  training  techniques  and 
procedures  for  improving  marksmanship  performance  via  ISMT.  The  Navy  uses 
marksmanship  simulators  at  shore  facilities  for  sustainment  qualification,  at  a  gun  range 
or  conducts  a  gun  shoot  on  the  flight  deck  while  underway  in  order  to  ensure  gun 
qualification  cards  are  current  for  all  watch  standers.  Surface  Warfare  Officer  School 
(SWOS)  in  Newport,  Rhode  Island  exposes  department  head  students  to  a  marksmanship 
simulator  that  goes  over  several  security  scenarios.  However,  it  is  uncommon  for  naval 
vessels  to  have  a  dedicated  compartment  for  a  simulator  to  determine  improvement  in 
watch  stander  marksmanship  performance.  Ships  compile  a  list  of  shooters  based  on 
expiration  of  gun  qualifications  and  periodically  make  the  flight  deck  ready  for  live-fire 
training  exercises  while  underway.  Any  practice  prior  to  a  gun  qualification  is 
nonexistent  due  to  ammunition  onboard,  facilities  available  and/or  time  constrains. 
Familiarity  with  weapon  capabilities  is  based  on  duty  section  training  and  it  is  uncommon 
to  include  dry  fire  training  at  the  same  time.  Figure  1  is  a  picture  the  author  took  of  his 
commanding  officer  during  a  pistol  qualification. 


Figure  1.  Commander  J.  L.  Killman,  Commanding  Officer  of  USS  Elrod  (2012- 
2013)  and  GMl  Clay  as  the  range  safety  officer  during  a  pistol 
qualification  on  April  10,  2013. 
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A  qualification  conducted  while  underway  adds  extra  challenges  to  a  sailor 
because  the  sea  state  may  create  signifieant  pitch  and  roll  that  will  affect  the  shooters 
stability  as  the  flight  deek  shifts.  The  targets  are  weighted  down  and  tied  off  however  the 
weather  eould  cause  the  targets  to  sway  if  the  wind  is  strong  enough.  The  shooters  must 
adapt  and  overeome  these  faetors  to  qualify.  By  using  simulation-based  training,  sailors 
ean  re-enforce  their  marksmanship  skill  with  repetitive  weapon  handling  drills  and 
reeeive  feedbaek  on  accuracy.  This  technology  can  be  used  to  add  training  value  to  a 
watch  stander  onboard  a  ship  regardless  of  geographic  location  or  time  in  the  current 
training  cycle. 

Commander  Naval  Surface  Forces  Atlantie  (COMNAVSURFLANT)  released  a 
naval  message,  with  date  time  group  of  R  271735ZFEB14,  for  a  small  arms  live  fire  and 
simulator  faeility  information  request.  Paragraph  1  of  this  message  reads 
(COMNAVSURFLANT,  2014). 

Feedbaek  from  the  fleet  indicates  a  need  to  analyze  the  capabilities  of 
navy  ranges  and  small  arms  weapons  simulators  (SAWS).  In  order  to 
determine  if  efficieneies  in  seheduling,  throughput,  operational  hours, 
physical  expansion  and  new  eonstruetion  of  small  arms  ranges  are 
required  to  meet  operational  requirements,  (p.  1) 

With  the  increase  in  operational  tempo,  the  U.S.  Navy  has  had  little  opportunity 
for  sailors  to  reeeive  training  and  practice  on  a  live  range  prior  to  qualifications.  As  a 
result,  there  has  been  limited  training  afforded  to  sailors  who  must  receive  a  passing 
qualifieation  seore  in  order  to  stand  a  foree  protection  watch  that  required  a  gun. 
COMNAVSURFLANT  further  states  (COMNAVSURFLANT,  2014): 

...Although  Commanders  have  done  an  excellent  job  qualifying 
personnel,  this  practiee  has  resulted  in  insufficient  data  to  make  informed 
operational,  budgetary,  and  resource  alloeation  decisions  with  regard  to 
range  capabilities,  limitations,  scheduling,  maintenance,  and  the 
optimization  of  training,  (p.  1) 

The  ISMT  is  a  portable  stand-alone  marksmanship  trainer  that  currently  is  used  to 
help  train  U.S.  Navy  and  Marines.  The  ISMT  is  designed  to  train  sailors  and  marines  in  a 
classroom  setting  and  uses  various  weapons  such  as  the  M9  and  M16A2  as  its  training 
weapons.  It  has  the  capability  to  train  up  to  four  individuals  and  provides  immediate 
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feedback  to  the  instractor  and  trainee  on  weapon  trigger  pull,  cant  position,  barrel 
movement,  rifle  butt  pressure,  tracing  of  the  muzzle  on  a  weapon  prior  to  and  after  a  shot, 
and  impact  grouping.  This  feedback  enables  the  participants  to  adjust  their  weapon 
accordingly  to  improve  accuracy.  More  importantly,  this  feedback  allows  the  instructor  to 
effectively  evaluate  the  individual’s  technique  because  the  muzzle  movement  is  recorded 
and  displayed  for  review  under  a  trace  profile  feature.  The  human  eye  may  not  be  able  to 
see  what  happens  to  the  weapons  when  the  participant  pulls  the  trigger.  This  tracking 
technology  allows  instructors  to  take  advantage  of  improving  the  feedback  to  the  shooter 
for  better  performance. 

The  focus  of  this  thesis  is  to  provide  data  regarding  the  improvement  of  training 
via  ISMT.  In  order  to  have  an  effective  training  program,  marksmanship  training  needs  to 
be  divided  into  three  phases:  learning  what  shooting  a  weapon  entails  (breathing,  etc.); 
understanding  the  basic  knowledge  of  weapons  handling,  techniques,  and  fundamentals; 
and  execution  of  skills  acquired  on  a  range  (Chung,  Delacruz,  Vries,  Bewley,  &  Baker, 
2006).  Gunners  mates  (GMs)  and  master  at  arms  (MAs)  are  generally  proficient  on  the 
first  phase  because  there  are  several  power  point  tutorials  and  training  instruction  that 
cover  what  shooting  entails.  Training  weapons  are  used  for  the  second  phase,  basic 
knowledge  of  weapons  handling,  techniques  and  fundamentals;  this  training  typically 
happens  a  few  months  prior  to  an  antiterrorism  or  force  protection  certification. 

The  third  phase,  execution  of  skills,  is  critical,  yet  is  the  most  lacking  in  the 
program  in  the  experience  of  the  author.  In  the  author’s  experience,  sailors  are  expected 
to  execute  skills  acquired  on  a  range  if  a  real-world  threat  presented  itself  However, 
sailors  only  handle  weapons  during  watch  turnover  on  duty  days  that  occur  once  or  twice 
a  week  depending  on  the  number  of  duty  sections.  Sailors  do  not  shoot  again  until  their 
qualification  is  about  to  expire  in  order  to  maintain  the  number  of  duty  sections.  As 
previously  stated,  sailors  that  fail  to  meet  the  marksman  category  are  sent  to  the  back  of 
the  line  for  a  second  attempt  to  qualify.  If  sailors  qualify  as  a  marksman  or  sharpshooter, 
they  are  expected  to  carry  out  their  duties  and  are  not  allowed  to  shoot  again.  There  is  no 
expectation  that  a  sailor  needs  to  perform  better  or  to  continue  to  practice  any  marksman 
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skills  and  techniques.  It  is  this  third  phase  in  which  simulation  training  may  be 
particularly  useful. 


B.  LITERATURE  REVIEW  ON  SIMULATED  MARKSMANSHIP 
TRAINING 

Below,  literature  regarding  the  use  of  simulation  in  marksmanship  training  is 
reviewed.  The  focus  of  the  literature  review  is  the  few  studies  that  have  compared 
simulation  training  in  marksmanship  to  live  training.  Emphasis  is  placed  on  studies  that 
specifically  examined  the  ISMT.  Baker  et  al.  (2004)  conducted  research  for  rifle 
marksmanship  performance  focusing  on  predicting  the  shooting  performance  of 
participants  utilizing  advance  distributed  learning  environments  (i.e.,  sensing  apparatus). 
Her  research  showed  evidence  to  suggest  a  base  knowledge  component  to  shooting 
performance  and  differences  in  knowledge  of  marksmanship.  Using  the  advanced 
distributed  learning  environment,  she  was  able  to  predict  the  participant’s  marksmanship 
scores  regardless  of  the  participant’s  prior  performance  scores  (Chung  et  al.,  2006). 

Scribner,  Wiley  and  Harper  (2007)  conducted  a  training  transfer  study  using 
dismounted  infantry  survivability  and  lethality  test-bed  (DISALT)  for  simulation-based 
training  to  determine  differences  between  the  dependent  performance  and  subjective 
measures  for  a  live  versus  simulated  pop-up  target  shooting  with  a  rifle.  They  used  12 
Army  soldiers  with  a  military  occupational  specialty  of  indirect  fire  dismounted 
infantryman.  In  their  experiment,  each  participant  was  exposed  to  ten  trials  in  a 
simulation  facility  and  10  trials  on  a  live  fire  range  (Scribner  et  al.,  2007).  Half  of  the  12 
Amy  soldiers  completed  trials  in  the  simulator  and  then  on  the  live  fire  range.  The  other 
half  completed  their  trials  in  the  opposite  order.  The  results  showed  no  significant 
difference  in  the  main  marksmanship  score  or  hit  percentages  between  the  two  different 
types  of  exercises.  In  other  words,  simulation  performance  was  equivalent  to  live  fire 
performance  on  main  measures  of  performance.  However,  differences  did  emerge  on 
more  subtle  measures  of  marksmanship  performance:  reaction  time  and  radial  aiming 
error.  Reaction  time  was  longer  and  the  radial  error  was  lower  in  the  simulated 
environment.  When  they  were  engaged  in  live  fire,  they  experience  more  of  the  elements 

when  compared  to  people  in  a  simulator  and  thus  react  more  quickly.  Human  attributes, 
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such  as  rifle  movement  with  a  live  weapon  and  rounds,  may  change  the  psyehologieal 
dynamie  of  the  seenario  (Scribner  et  ah,  2007).  Beeause  of  these  results,  this  thesis 
foeused  on  the  use  of  a  simulated  9mm  Beretta  to  prevent  this  human  attribute  from 
affeeting  the  study  of  standard  Navy  marksmanship  training  and  simulation-based 
marksmanship  training. 

Two  studies  that  speeifieally  foeused  on  the  use  of  ISMT  for  training  were  written 
by  Major  William  Yates  (2004)  and  by  Lieutenants  Timothy  Jensen  and  John  Woodson 
(2012).  Both  studies  examine  training  transfer,  but  Yates  used  a  M16A  rifle  while  Jensen 
and  Woodson  used  a  9mm  Beretta.  Yates’s  study  contained  two  experiments  the  first 
experiment  evaluated  and  quantified  the  effeetiveness  of  the  ISMT  in  training  noviee 
shooters  in  the  teehnical  operation  of  the  M16A  serviee  rifle.  The  second  experiment 
assessed  whether  task  performanee  in  the  ISMT  is  of  predietive  value  with  respect  to  the 
actual  task  performance  using  live  ammunition  on  the  Marine  Corps’  standard  known 
distance  course  of  fire  (Yates,  2004). 

The  partieipants  for  the  first  experiment  were  an  all-male  platoon  of  Marine  Corp 
reeruits  at  Edson  Range  on  Camp  Pendleton  Marine  Corps  Base,  California.  The  eontrol 
group  had  no  knowledge  of  the  ISMT  and  eondueted  training  with  a  technique  referred  to 
as  “snapping-in.”  Snapping-in  ordered  the  partieipants  to  assume  the  shooting  position, 
aim  an  unloaded  rifle  with  an  empty  magazine  and  “dry-fire.”  Dry-firing  is  the  actuation 
of  the  trigger  without  any  ammunition  being  loaded  in  the  weapon  (Yates,  2004).  The  test 
group  had  approximately  nine  hours  of  training  with  ISMT.  The  control  group  and  test 
group  both  utilized  the  same  instruetor  and  had  55  partieipants  in  each  group. 

The  second  experiment  provided  insight  on  the  aequisition  of  marksmen  skill  with 

repetitive  practice  on  the  ISMT  without  the  presence  of  a  skilled  eoach.  The  28 

partieipants  included  civilians  with  no  affiliation  to  the  Department  of  Defense  (DOD) 

and  military  offieers  assigned  as  students  and  civilian  employees  at  the  Naval 

Postgraduate  Sehool  (Yates,  2004).  Thirteen  subjects  completed  at  least  seven  trials.  The 

other  15  partieipants  completed  four  or  fewer  trials.  Every  partieipant  was  eharacterized 

as  either  trained  or  untrained  based  on  any  formal  marksmanship  instruction.  This 

experiment  measured  the  eonsistency  of  their  aim  based  on  the  average  diameter  of  their 
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shot  groupings.  They  were  expected  to  fire  two  strings  of  three  rounds  each  and  the  third 
string  fired  four  rounds.  The  furthest  outlying  round  of  the  third  string  of  fire  was 
discarded.  These  participants  lacked  the  advice  of  a  skilled  coach  but  were  briefed  on 
how  to  use  the  rifle,  fundamentals  of  good  shooting  position,  weapon  support,  sight 
picture  and  alignment,  trigger  squeeze  and  breather  control  (Yates,  2004). 

The  results  from  the  first  experiment  show  equivalent  performance  in 
marksmanship  scores  of  recruits  trained  on  ISMT  versus  recruits  in  a  control  group 
without  ISMT  training.  It  is  important  to  note  that  the  test  group  experienced  foul 
weather  condition  on  the  last  three  days  that  led  up  to  the  qualification  day,  while  the 
control  group  experienced  optimal  weather  condition  all  five  days  leading  up  to 
qualification  day.  Yates’s  study  was  able  to  prove  that  subjects  with  ISMT  training  does 
not  result  in  negative  training  transfer  (2004).  Additionally,  there  was  a  positive 
correlation  between  simulated  qualification  COF  scores  and  live  fire  qualification  COF 
scores  (Yates,  2004).  The  second  experiment  found  no  evidence  to  support  that  untrained 
participants  produced  steady  improvement  using  ISMT  without  a  coach  (Yates,  2004). 
However,  a  trained  shooter  retained  proficiency  and  showed  small  increments  of  steady 
improvement  without  a  coach  (Yates,  2004). 

The  results  above  are  somewhat  ambiguous,  but  they  indicate  that  simulator 
training  can  lead  to  equivalent  performance  as  live  fire  training  for  certain  tasks,  and 
therefore,  may  be  a  good  alternative  when  live  fire  training  is  unavailable.  People  may 
not  be  ready  to  replace  live  fire  training  with  simulated  marksmanship  training  but  these 
advances  in  marksmanship  simulators  are  vital  to  improving  the  performance  of  a  shooter 
through  part  task  training  in  a  virtual  environment.  Of  note,  the  above  experiments  used  a 
rifle  as  the  weapon  of  choice.  A  comparison  of  simulator  and  live  fire  training  on  other 
weapons,  such  as  pistols  is  needed. 

Jenson  and  Woodson  (2012)  informally  interviewed  approximately  50  Navy 

officers  that  served  in  billets  located  near  the  41  ISMT  systems  across  the  fleet  and  only 

one  person  knew  ISMT  existed.  This  finding  pinpoints  the  current  problem  in  Navy 

marksmanship  training.  By  instruction,  sailors  are  expected  to  use  simulators  to  aid  in 

marksmanship  training,  yet  the  Navy  commands  are  not  utilizing  available  assets  or 
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ISMT  training  facilities  are  not  supporting  the  fleet.  The  purpose  of  their  study  explored 
whether  simulation-based  marksmanship  training  with  ISMT  transfers  to  live  fire  and 
provided  insight  into  how  long  beneficial  effects  of  simulation  training  lasted  (Jenson  & 
Woodson,  2012).  Their  study  used  34  participants  of  active  duty  military  or  faculty  at 
Naval  Postgraduate  School  in  a  two  (control  versus  simulated  training)  by  two  (two  or 
four  week  gap  between  training  and  live  fire  qualification).  Thus,  there  were  four  groups: 
two  control  groups  trained  with  standard  Navy  marksmanship  training  and  the  two  test 
groups  trained  with  simulation  training  on  ISMT.  One  control  and  test  group  received  no 
additional  training  after  two  weeks,  and  the  other  control  and  test  group  received  no 
additional  training  after  four  weeks.  All  four  groups  conducted  a  final  live  fire  Navy 
handgun  qualification  course  that  determined  retention  of  marksmanship  knowledge. 
Importantly,  participants  were  blocked  into  the  groups  by  self-reported  previous 
marksmanship  experience. 

The  training  programs  provided  were  baseline  training,  control  group 
marksmanship  fundamental  training  and  simulation  group  marksmanship  fundamental 
training.  All  participants  received  baseline  training  and  covered  safety,  weapon 
conditions  and  commands,  training  environment  familiarity,  and  approved  firing  position. 
The  control  groups  marksmanship  fundamentals  covered  dry  fire  procedures,  aiming, 
sight  alignment,  proper  grip  and  control,  trigger  pull,  breather  control  and  practiced  dry 
fire  training.  The  simulation  group  marksmanship  training  covered  the  same  topic  but 
excluded  the  dry  fire  procedures  because  the  simulated  9mm  Beretta  was  connected  to 
ISMT  and  a  carbon  dioxide  (CO^)bottle  in  order  to  provide  video  and  audio  feedback  and 
simulated  recoil. 

The  results  comparing  performance  at  baseline  qualification  to  live  fire 
qualification  found  that  simulation  groups  showed  significantly  less  degradation  in  mean 
point  of  impact  (MPI)  performance  than  the  control  group.  The  MPI  provides  a  more 
accurate  measure  of  marksmanship  performance  than  score  and  therefore  may  be  more 
indicative  of  how  well  a  watch  stander  would  perform  in  an  actual  threat  situation 
(Jenson  &  Woodson,  2012).  It  is  important  to  note  that  the  simulation  group  achieved 
better  MPI  performance  than  the  control  group  at  the  seven-  and  15-yard  distance  during 
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post  training  and  performed  better  at  the  three-yard  distance  for  live  fire.  The  results  on 
retention  show  no  significant  difference  in  either  two  or  four  week  gaps.  All  groups 
showed  a  pattern  of  performance  degradation  since  their  post-training  but  also  showed 
improvement  from  their  baseline  qualification  to  the  live  fire  qualification. 

The  Navy  could  benefit  from  a  study  that  compares  current  naval  marksmanship 
training  and  simulation-based  marksmanship  training.  The  experiment  presented  in  this 
thesis  assessed  whether  ISMT  can  be  used  as  an  effective  part-task  trainer  to  improve 
marksmanship  performance  for  the  9mm  Berretta  on  the  standard  navy  handgun 
qualification  COF  compared  to  current  naval  marksmanship  training.  Like  the  Jenson  and 
Woodson  (2012)  study,  participants  in  this  thesis  were  categorized  based  on  their 
qualification  score  in  accordance  with  standard  Navy  Handgun  Qualification  COF  (Chief 
of  Naval  Operations,  2009).  The  categories  are  as  follows:  novices,  marksman, 
sharpshooter,  or  expert  prior  to  training  on  the  ISMT  (Clark,  2010;  Chief  of  Naval 
Operations,  2009).  This  thesis  also  assessed  how  well  participants  retained 
marksmanship  skills  over  a  one-week  period.  Pistol  marksmanship  training  was  the 
focus  of  this  experiment  because  the  majority  of  the  naval  security  watch  standers  do  not 
carry  rifles.  The  next  section  explains  the  details  of  the  training  programs  created  for  this 
experiment. 

C.  MARKSMANSHIP  TRAINING  BACKGROUND 
1.  Training  Requirement 

The  current  naval  training  requirements  are  specifically  addressed  in  the  small 
arms  training  and  qualificaiton  instruction  within  enclosure  three,  paragraph  five  (Chief 
of  Naval  Operations,  2009). 

Before  each  qualification  fire  and  sustainment  training  session,  all 
shooters  shall  receive  instruction  on  marksmanship,  safety  and  weapon 
familiarization...  Shooters  shall  also  practice  drawing  from  the  holster, 
and  must  demonstrate  the  ability  to  safely  handle  and  present  the  weapon 
to  the  instructor  before  live  firing.  Shooters  must  also  demonstrate 
knowledge  of  the  four  general  safety  rules,  weapons  commands,  and 
weapon  condition  codes  before  live  firing. . .  (p.  2) 
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All  of  the  required  training  can  be  found  in  the  separate  enclosures  of  the  small 
arms  training  and  qualificaiton  instruction  (Chief  of  Naval  Operations,  2009).  Enclosure 
one  covers  the  range  regulations,  general  safety  rules,  general  range  safety  rules,  range 
operations,  weapons  commands,  pistol  safety  rules,  and  training  time  out.  Enclosure  two 
covers  the  procedure  to  properly  conduct  dry  fire  training  and  enclosure  three  explains 
the  qualification  criteria  for  pistols  and  the  navy  handgun  qualification  course  of  fire 
(Chief  of  Naval  Operations,  2009).  Enclosure  three  was  used  to  ensure  that  the 
qualification  course  of  fire  (COF)  scenario  created  in  ISMT  was  within  the  current  naval 
standards.  This  thesis  also  used  the  Force  Protection  Weapons  Handling  Standard 
Procedures  and  Guidelines  (U.S  Fleet  Forces  Command,  2003),  Pistol  Marksmanship 
(Headquarters,  United  States  Marine  Corps,  2003),  the  Combat  Training  with  Pistold,  M9 
and  Mil  (Headquarters,  Department  of  the  Army,  2003).  These  reference  were  used  to 
enhance  the  training  provided  by  the  small  arms  training  and  qualificaiton  instruction 
(Chief  of  Naval  Operations,  2009). 

D.  RESEARCH  QUESTIONS  AND  HYPOTHESIS 

I.  Question  One 

Can  the  ISMT  be  used  as  an  effective  part-task  trainer  to  improve  marksmanship 
performance  for  the  9mm  Berretta  on  the  standard  Navy  handgun  qualification  COF? 

Null  hypothesis  (HO):  Participants  who  receive  simulation-based  marksmanship 
training  in  ISMT  will  have  the  same  amount  of  improvement,  from  the  baseline 
qualification  to  the  training  qualification,  in  marksmanship  performance  and  scores  on 
the  standard  navy  handgun  qualification  COF  than  the  participants  who  received  current 
naval  marksmanship  training.  The  control  group  and  the  experimental  group  will  have 
the  same  level  of  improvement  regardless  of  the  marksmanship  training  provided. 

Alternate  hypothesis  (HA):  Participants  who  receive  simulation-based 
marksmanship  training  in  ISMT  will  have  a  greater  improvement,  from  the  baseline 
qualification  to  the  training  qualification,  in  marksmanship  performance  and  scores  on 
the  standard  navy  handgun  qualification  COF  than  the  participants  who  received  current 
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naval  marksmanship  training.  The  experimental  group  will  improve  more  than  the  control 
group  as  a  result  of  the  marksmanship  training  provided. 

2.  Question  Two 

If  a  participant  in  the  experimental  (simulation)  group  does  not  pass  the  baseline 
standard  navy  handgun  qualification  COF,  will  they  have  a  better  chance  of  meeting  the 
marksmanship  standard  after  receiving  simulation-based  training  in  ISMT  than  a 
participant  in  the  control  group? 

HO:  Among  those  who  do  not  meet  the  qualification  standard  for  the  standard 
navy  handgun  qualification  COF  at  baseline,  there  will  be  an  equal  number  of 
participants  in  each  group  who  do  not  meet  the  qualification  standard  after  their 
simulated  marksmanship  training.  Those  in  control  group  and  the  experimental  group 
that  fail  the  baseline  COF  will  have  the  same  level  of  improvement  regardless  of  the 
marksmanship  training  provided. 

HA:  Among  those  who  do  not  meet  the  qualification  standard  for  the  standard 
navy  handgun  qualification  COF  at  baseline,  a  greater  number  of  participants  in  the 
experimental  group  will  reach  the  qualification  standard  after  simulation-based 
marksmanship  training  than  in  the  control  group.  Those  in  the  experimental  group  that 
fail  the  baseline  COF  will  improvement  more  than  those  control  group  that  fail  the 
baseline  COF  as  a  result  of  the  marksmanship  training  provided. 

3.  Question  Three 

Will  participants  in  the  experimental  (simulation)  group  have  a  better  chance  of 
retaining  what  they  learned  after  one  week  of  no  instruction,  than  participants  in  the 
control  group? 

HO:  Participants  in  each  group  will  be  equally  likely  to  maintain  their 
marksmanship  performance  and  scores  on  the  retention  standard  navy  handgun 
qualification  COF  one  week  after  the  training  qualification.  Those  in  the  control  group 
and  the  experimental  group  will  retain  the  same  level  of  marksmanship  skill  after  a  week 
of  no  training. 
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HA:  Participants  in  the  experimental  group  will  be  more  likely  than  those  in  the 
control  group  to  maintain  their  marksmanship  performanee  and  scores  on  the  retention 
standard  navy  handgun  qualification  COF  one  week  after  the  training  qualification. 
Those  in  the  experimental  group  will  retain  a  higher  level  of  marksmanship  skill  than  the 
control  group  after  a  week  of  no  training. 

4.  Overview  of  Thesis 

A  demographie  survey  was  administered  to  all  partieipants  prior  to  the  beginning 
of  the  experiment  (see  Appendix  H).  A  baseline  qualifieation  assessed  their  current  level 
of  marksmanship  performance.  The  participants’  baseline  scores  along  with  their  reported 
previous  experience  on  the  demographic  survey  were  used  to  block  the  partieipants  into 
the  control  and  experimental  groups.  The  performance  measurements  and  seore  on  the 
baseline  qualifieation  were  recorded  before  the  participants  received  any  simulated 
marksmanship  training.  The  second  handgun  qualifications  were  administered  after  the 
control  and  experimental  groups  reeeive  their  designated  simulated  marksmanship 
training  to  establish  any  main  effects  from  their  baseline.  Partieipants  did  not  reeeive 
additional  training  for  a  week  and  the  instructor  administered  a  final  handgun 
qualifieation  to  establish  retention  of  marksmanship  skills. 

E.  THESIS  ORGANIZATION 

The  introduction  chapter  provides  the  rationale  behind  this  thesis.  The  literature 
review  gives  focused  overview  of  studies  in  the  field  of  marksmanship  and  how  previous 
work  lead  up  to  this  thesis.  The  baekground  information  on  marksmanship  training  is  an 
explanation  of  the  eurrent  naval  standards,  and  naval  training  requirements  .The  researeh 
questions  and  corresponding  hypotheses  are  listed,  as  well  as  an  overview  of  the 
experiment. 

The  methods  chapter  explains  the  researeh  design,  demographic  characteristics  of 

the  partieipants,  equipment  and  seenarios,  main  measures,  and  proeedures.  The  research 

design  section  provides  a  depiction  of  independent  and  dependent  variables  followed  by  a 

list  of  controls  used  to  conduct  the  experiment.  The  next  section  goes  into  the  process  of 

how  partieipants  were  recruited  and  a  description  of  how  they  were  split  up  into  either  the 
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control  group  or  experimental  group.  The  methods  ehapter  closes  with  an  explanation  of 
the  proeedures  to  eonduct  the  experiment  and  provide  enough  detail  for  future  thesis 
students  to  repeat  the  same  work. 

The  results  chapter  provides  the  results  of  the  experiment  starting  with  a  broad 
look  at  the  descriptive  statisties  and  then  deseribing  the  results  for  each  hypothesis.  The 
diseussion  ehapter  leads  into  an  interpretation  of  the  findings  based  on  the  significant  and 
non- significant  results. 

The  last  chapter  is  the  eonelusions  and  reeommendations.  This  ehapter  focuses  on 
any  shorteomings  and  how  to  improve  future  work.  This  chapter  also  identifies  all  ISMT 
technical  difficulties  with  the  software  while  creating  the  naval  standard  handgun 
qualifieations  scenarios  and  the  malfunctions  with  seven  of  the  M9  Berettas. 
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II.  METHOD 


A.  STUDY  OBJECTIVES 

The  objective  of  this  thesis  was  to  determine  any  changes  on  marksmanship 
performance  based  on  the  type  of  training  a  participant  received.  The  ISMT  was  used  as  a 
part  task  trainer  to  help  participants  in  the  experimental  group  learn  how  to  fire  a  pistol 
one  step  at  a  time.  Dry  fire  training  was  conducted  with  the  control  group  for  the  same 
purpose  of  teaching  marksmanship.  The  experiment  determined  if  simulation-based 
training  conducted  in  ISMT  was  able  to  increase  marksmanship  performance  to  a  greater 
extent  than  a  training  program  that  meets  current  naval  marksmanship  requirements. 

1.  Research  Design 

The  experimental  design  for  this  thesis  utilized  between-groups  comparisons  in 
which  participants  were  blocked  by  previous  marksmanship  experience.  The  independent 
variables  were  the  different  training  programs:  the  control  group  and  experimental  group. 
The  control  group  received  current  naval  marksmanship  training  that  went  over 
fundamentals  of  marksmanship  followed  by  untimed  dry  fire  and  a  practice  qualification 
that  implemented  time  limits.  The  experimental  group  received  simulation-based 
marksmanship  training  that  consisted  of  fundamentals  of  marksmanship  followed  by 
untimed  simulation  practice  and  a  practice  qualification  that  implemented  time  limits. 

2.  Training  Methods  Used 

A  brief  overview  of  the  procedure  was  administered  to  the  participants  before 
they  received  what  was  referred  to  as  baseline  training  in  this  thesis.  The  participant  then 
received  their  baseline  qualification.  Enclosure  one  of  the  small  arms  training  and 
qualificaiton  instruction  (Chief  of  Naval  Operations,  2009)  was  used  to  ensure  that  the 
safety  training  requirements  were  met.  The  purpose  of  this  training  was  to  establish  a 
basic  level  of  knowledge  for  participants  that  have  never  fired  a  weapon.  All  the 
participants  were  from  the  Naval  Postgraduate  School  (NPS);  however  the  level  of 
marksmanship  skill  at  the  beginning  of  the  experiment  will  drastically  vary  depending  on 
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if  a  participant  is  a  civilian  (either  a  NFS  student  or  facility  and  staff)  or  active  duty 
military  (assigned  to  NFS).  The  participants  that  were  active  duty  could  still  have  a  lot  of 
variability  in  their  skill  level  because  of  the  different  services. 

With  all  the  many  possibilities,  the  baseline  training  program  ensured  that  at  a 
minimum,  everyone  started  with  a  basic  understanding  of  the  requirements  for  this 
experiment.  The  training  covered  how  to  use  the  M9  Berretta,  and  explained  the  specific 
weapon  commands  used  in  the  experiment.  Refer  to  Table  1  for  the  content  of  the  entire 
baseline  training  program. 


Baseline  Training 

General  Range  Safety  Rules 
General  Safety  Rules 
Weapon  Conditions 
Weapon  Commands 
Pistol  Safety  Rules 
Ready  Line 
Firing  Line 
Range  Operations 
Training  Time  Out 
Loading  the  Pistol 
Making  the  Pistol  Ready 
Fire 

Cease  Fire 

Unload,  Showing  the  Pistol  Clear 
Dry  Reload 
Isosceles  only 
Isosceles  Standing  Position 
Isosceles  High  Kneeling  Position 

Table  1.  Baseline  Training  Frogram 

It  is  important  to  note  that  no  marksmanship  fundamental  training  was  provided  because 
the  purpose  of  the  baseline  qualification  was  to  capture  the  level  of  marksmanship  skill 
that  each  participant  had  at  the  beginning  of  the  experiment. 

The  second  training  programs  that  the  participants  received  were  used  to 
determine  if  there  were  any  changes  from  the  baseline  qualification.  The  second  chapter 
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of  both  the  Force  Protection  Weapons  Handling  Standard  Procedures  and  Guidelines 
(U.S  Fleet  Forces  Command,  2003),  and  the  Pistol  Marksmanship  (Headquarters,  United 
States  Marine  Corps,  2003)  were  also  used  to  establish  the  baseline  training  program.  It  is 
also  essential  to  note  that  the  last  part  of  the  baseline  training  program  specifically 
addressed  the  isosceles  firing  positions.  Refer  to  Figure  2  for  a  picture  of  the  isosceles 
standing  position  and  refer  to  Figure  3  for  a  picture  of  the  isosceles  high  kneeling 
position. 


Figure  2.  Isosceles  Standing  Position  from  Pistol  Marksmanship  (from  Headquarters, 

United  States  Marine  Corps,  2003) 
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Figure  3.  Isosceles  High  Kneeling  Position  from  Pistol  Marksmanship  (from 
Headquarters,  United  States  Marine  Corps,  2003) 

According  to  the  small  arms  training  and  qualificaiton  instruction  (Chief  of  Naval 
Operations,  2009)  the  isosceles  and  Weaver  firing  positions  were  acceptable.  The  Weaver 
stance  was  not  allowed  in  this  thesis  because  of  the  difficulty  in  maintaining  controls  for 
the  experiment.  The  instructors  would  have  had  a  difficult  time  trying  to  ensure  that 
every  participant  assumed  the  exact  same  Weaver  standing  position  every  single  time. 
The  Weaver  high  kneeling  position  would  have  provided  so  much  support  that  it  would 
have  been  more  difficult  to  determine  if  the  participant  was  actually  utilizing  the  proper 
breathing  technique,  sight  picture  and  sight  alignment.  The  isosceles  high  kneel  position 
would  have  provided  enough  support  for  the  participant  to  maintain  their  balance.  It 
would  have  also  been  just  enough  instability  to  make  it  easier  to  determine  if  the 
participant  used  proper  marksmanship  fundamentals.  Refer  to  Figure  4  for  a  picture  of  the 
Weaver  standing  position  and  Figure  5  for  a  picture  of  the  Weaver  high  kneeling 
position. 
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Figure  4.  Weaver  Standing  Position  from  Pistol  Marksmanship  (from  Headquarters, 

United  States  Marine  Corps,  2003) 
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Figure  5.  Weaver  High  Kneeling  Position  from  Pistol  Marksmanship  (from 
Headquarters,  United  States  Marine  Corps,  2003) 

The  control  group  training  program  was  created  by  using  enclosure  two  of  the 
small  arms  training  and  qualificaiton  instruction  (Chief  of  Naval  Operations,  2009), 
Chapter  2  of  the  Force  Protection  Weapons  Handling  Standard  Procedures  and 
Guidelines  (U.S  Fleet  Forces  Command,  2003),  and  Chapter  3  of  the  Pistol 
Marksmanship  (Headquarters,  United  States  Marine  Corps,  2003).  The  participants  were 
allowed  to  handle  the  M9  Berretta  and  follow  along  with  the  recorded  lessons.  It  was 
important  that  the  participants  were  afforded  the  opportunity  to  practice  without  any  time 
constraints.  During  the  lecture,  the  participants  focused  on  slow,  smooth  and  methodical 
motions  when  drawing  and  dry  firing  the  pistol.  The  participants  in  the  control  group 
received  18  minutes  of  recorded  lecture.  Refer  to  Table  2  for  a  complete  list  of  the  items 
covered  in  the  lecture. 
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Control  Group  Training _ 

Dry  Fire  Safety  Training 
Drawing  Process 
Grab 
Draw 
Smack 
LookSqueez 
Holstering 

Introduction  to  Fundamentals 
Aiming 

Sight  Alignment 
Establishing  Sight  Alignment 
Grip  with  respect  to  {WRT)  Aiming 
Controlled  MuscularTension 
Sight  Picture 

Relationship  Betweentthe  Eye  and  the  Sights 
Trigger  Control 

Sight  Alignment  and  Trigger  Control 
GripWRTTriggerControl 
Trigger  Finger  Placement 
Uninterrupted  Trigger  Control 
Interrupted  Trigger  Control 
Breath  Control 
Begin  Dry  Fire  Training 
Single  Action 

Five  to  Ten  Times  Repitition 
Start  Adding  Time  Limits 
Draw  Sweep  Safety  Dry  Fire 

Table  2.  Control  Group,  Marksmanship  Fundamental  Training  Program 

A  practice  qualification  was  administered  to  the  control  group  after  completing 
the  marksmanship  fundamentals  lecture.  The  M9  Berretta  was  disconnected  from  the 
ISMT  which  prevented  the  use  of  carbon  dioxide  (CO^)  and  only  dry  fire  training 
techniques  was  used.  The  purpose  of  this  practice  was  to  implement  time  constrains  and 
to  build  the  participants’  expectations  towards  a  navy  handgun  qualification.  There  was  a 
pause  between  trials  at  the  three,  seven  and  15-yard  line.  This  pause  was  provided  to  the 
instructor  in  case  any  training  was  required  for  the  participant  to  maximize  training 
needs.  Upon  completion  of  the  practice  qualification,  no  further  training  was  administer 

for  the  duration  of  their  involvement  with  the  experiment.  A  second  qualification  was 
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administered  to  the  participants  in  the  control  group  to  determine  any  main  effects  as  a 
result  of  the  current  naval  marksmanship  training  program.  After  a  week  of  no  further 
training  the  participants  came  back  on  the  same  day  of  the  week  and  time  of  day  for  a 
third  qualification.  This  qualification  was  done  to  determine  if  any  of  the  participants 
retained  any  marksmanship  skill. 

3.  UseofISMT 

The  experimental  group  training  program  was  created  in  a  similar  manner,  but 
without  using  enclosure  2  of  the  small  arms  training  and  qualificaiton  instruction  (Chief 
of  Naval  Operations,  2009).  Instead  of  conducting  dry  fire  during  the  marksmanship 
fundamentals  training  lecture,  they  received  simulation-based  training  that  provided 
instant  feedback  from  the  ISMT.  The  current  simulator  training  requirements  for  the  navy 
are  specifically  addressed  in  the  small  arms  training  and  qualificaiton  instruction  within 
Enclosure  three,  paragraph  5  (Chief  of  Naval  Operations,  2009). 

Where  available,  simulator  training  shall  include  marksmanship  training 
drills  to  assist  instructors  in  identifying  and  correcting  personnel  shooter 
fundamental  errors.  Courses  of  fire  should  be  programmed  into  the 
simulator  to  provide  personnel  the  opportunity  to  simulate  firing  the  COF 
prior  to  live  fire  operations,  (p.  3) 

The  participants  received  16  minutes  of  recorded  lecture.  The  control  group 
lecture  was  2  minutes  longer  because  it  took  longer  to  read  the  requirements  for  dry  fire 
versus  simulation  training  requirements.  The  difference  in  time  did  not  cause  any  drastic 
differences  between  the  two  groups.  The  experimental  group  replaced  the  time  lost  in  the 
lecture  with  time  to  review  trace  profiles  from  their  baseline  qualification  during  the 
pauses  in  between  stages  of  the  practice  qualification.  Refer  to  Table  3  for  a  complete  list 
of  the  items  covered  in  the  experimental  group,  marksmanship  fundamentals  lecture. 
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Simulation  Group  Training 
Simulation  Safety  Training 
Drawing  Process 
Grab 
Draw 
Smack 
LookSqueez 
Holstering 

Introduction  to  Fundamentals 
Aiming 

Sight  Alignment 
Establishing  Sight  Alignment 
Grip  with  respect  to  {WRT)  Aiming 
Controlled  MuscularTension 
Sight  Picture 

Relationship  Betweentthe  Eye  and  the  Sights 
Trigger  Control 

Sight  Alignment  and  Trigger  Control 
Grip  WRT  Trigger  Control 
Trigger  Finger  Placement 
Uninterrupted  Trigger  Control 
Interrupted  Trigger  Control 
Breath  Control 
Begin  Simulation  Training 

Table  3.  Experimental  Group,  Marksmanship  Fundamental  Training  Program 

The  partieipants  were  allowed  to  handle  the  M9  Berretta  and  follow  along  with 

the  recorded  lessons.  During  the  lecture,  the  participants  focused  on  slow,  smooth  and 

methodical  motions  when  drawing  and  dry  firing  the  pistol.  The  simulation  group 

received  immediate  feedback  from  the  ISMT  because  the  weapon  displayed  where  the 

shots  were  fired  on  the  target.  They  experienced  the  simulated  recoil  from  the  CO^  tanks 

and  the  ISMT  simulated  the  sound  of  a  pistol  being  fire  as  audio  feed  back.  The  training 

scenario  created  for  this  phase  of  the  training  is  called  unlimited  practice  and  its  purpose 

is  to  provide  the  participants  with  information  identifying  which  shot  was  the  last  one 

fired  and  displayed  a  value  associated  with  that  last  shot.  The  scenario  permanently 

displayed  three  targets  to  the  participant  to  represent  the  actual  size  of  the  three,  seven 

and  15  yard  line  targets.  The  control  group  saw  the  same  scenario  for  the  same  phase  of 

their  dry  fire  training  except  the  M9  Berretta  was  disconnected  from  the  ISMT. 
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A  practice  qualification  was  administered  to  the  simulation  group  upon 
completion  of  the  marksmanship  fundamentals  training  leeture.  The  practice  qualification 
paused  in  between  the  three,  seven  and  fifteen  yard  line  stages  of  fire.  Unlike  the  eontrol 
group,  the  simulation  group  had  an  opportunity  to  review  48  traee  profile  pictures  before 
they  shot  the  practice  qualifieation.  These  traee  profile  pietures  were  recorded  after  their 
baseline  qualification.  The  partieipant  viewed  all  twelve  shots  fired  at  the  three-yard  line 
and  tried,  with  the  instructor,  to  gather  any  information  addressing  marksmanship 
fundamental  errors.  Then  the  partieipants  shoot  the  three-yard  line  stage  of  the  praetiee 
qualifieation.  The  ISMT  seenario  was  paused  to  give  the  instruetor  time  to  present  the 
next  12  trace  profiles  pictures  of  shots  fired  at  the  seven  yard  line  from  the  baseline 
qualifieation.  After  the  partieipant  learned  from  the  instruetor’ s  comments  about  the  traee 
profile  pietures  they  shot  the  seven-yard  line  stage  of  the  practice  qualifieation.  The  same 
proeess  applied  for  the  pause  between  the  seven  and  15-yard  line. 

Upon  completion  of  the  praetiee  qualifieation,  the  participant  had  an  opportunity 
to  review  the  three-,  seven-  and  15-yard  line  shot  group  data  for  the  praetiee  qualifieation 
they  just  fired.  This  data  provided  feedback  of  their  performance  and  the  instructor 
identified  any  errors  that  can  be  addressed  before  the  seeond  qualification  was 
administered  to  the  participant.  Once  the  seeond  qualifieation  began  the  partieipant  no 
longer  received  any  assistance  from  the  instructor  and  had  to  rely  of  what  they  learned. 
After  a  week  of  no  further  training  the  participants  came  baek  on  the  same  day  and  time 
for  a  third  qualification.  This  qualification  session  was  administered  to  determine 
partieipants’  level  of  marksmanship  skill  retention. 

All  participants  completed  three  qualifieations.  The  first  provided  baseline 
marksmanship  skills.  The  second  was  used  to  determine  the  effect  of  the  training.  A  third 
qualifieation  was  administered  to  both  groups  exactly  one  week  after  the  completion  of 
training  to  determine  if  the  participants  retained  their  marksmanship  skills.  The 
dependent  variables  were  the  various  marksmanship  performance  data  that  ISMT 
provided  at  the  end  of  eaeh  qualification.  Marksmanship  performance  was  measured  by 
diameter  size  of  average  shot  group,  diameter  size  of  shot  group  for  the  three,  seven  and 
15  yard  line,  length  of  mean  point  of  impact  to  center  zeroing  point  (MPI  to  CZP)  for  the 
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three-,  seven-  and  15-yard  line,  and  scores  on  the  standard  navy  handgun  qualification 
COF. 


Research  Des%M 


Controls 

55  Minutes  Max 

per  session 

Recorded 

Course  of  Fire 

lnstru>ctions 

Isosceles 

Standing 

Position 

Isosceles 

High 

Kneeling 

Position 

IWeek 

Between  2nd 

and  3rd  dual 

Baseline  and 

Fundamentals 

Training 

Security- 

Belt  with 

Holster 

Firing  Line 

M9  Beretta 

Independent 

Variable 

Current  Naval 
Marksmanship 
Training 

Simulation 

Based 

Marksmanship 

Training 

Dependent 

Variable 

Average  Shot 
Group  Size 

Qualifi-cation 

Score 

Skill  Level 

Rating 

3  Yard  Line 

Shot  Group 

Size 

7  Yard  Line 

Shot  Group 

Size 

15  Yard  Line 

Shot  Group 

Size 

3  Yard  Line 

MPIto  CZP 

7  Yard  Line 

MPI  to  CZP 

15  Yard  Line 

MPI  to  CZP 

Table  4.  Research  Framework 


4.  Standardization  of  Methods  and  Reduction  of  Confounds 

There  were  several  controls  in  place  to  ensure  standardization  among  the 
participants  and  to  prevent  any  foreseeable  confounds.  Refer  to  Table  4  for  a  synopsis  of 
the  main  variables  of  interest,  and  the  factors  that  were  controlled  in  the  experiment.  The 
majority  of  the  participants  were  students  and  the  maximum  allotted  time  per  session  was 
55  minutes.  This  time  duration  allowed  students  to  participate  in  the  study  without 
interfering  with  any  class  schedule.  To  ensure  that  one  group  did  not  get  an  unfair 
advantage,  close  attention  was  paid  to  the  amount  of  training  time. 

Additionally,  the  author  conducted  a  class  project  (i.e.,  a  pilot  study)  to  learn  how 
to  use  ISMT  and  to  refine  the  experiment  for  this  thesis.  In  the  class  project,  the 
instructors  read  the  instruction  for  the  qualification  COF  and  conducted  the 
marksmanship  training  in  between  the  baseline  (first)  and  training  (second)  qualification. 
A  retention  (third)  qualification  was  not  part  of  the  class  project.  The  author  determined 
that  recording  all  the  instructions  for  the  COF  and  all  the  marksmanship  stimulus  training 
ensured  that  the  participants  were  not  affected  by  an  inconsistent  instructor.  The  ability  to 
repeat  the  same  playlist  ensured  that  one  group  did  not  have  an  advantage  over  the  other 
because  of  different  teaching  styles.  Prerecorded  media  files  also  assisted  the  instructor 
with  managing  fatigue  and  lose  of  vigilance. 
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The  decision  to  make  every  participant  utilize  the  isosceles  firing  positions  was  an 
attempt  to  avoid  any  confounds  between  different  stances.  It  was  difficult  to  say  that  a 
person  does  not  have  an  advantage  over  another  simply  because  one  person  decided  to 
use  a  Weaver  firing  position  instead  of  the  isosceles  firing  position.  The  isosceles 
standing  position  was  easier  and  ensured  that  every  participant  was  consistent.  The 
shoulders  had  to  be  square  to  the  target  at  all  times  and  the  feet  had  to  be  approximately 
shoulder  width  apart  every  time.  It  was  very  difficult  to  ensure  that  every  participant  had 
the  same  Weaver  standing  position,  because  there  was  no  way  to  determine  that  each 
person  had  their  body  turned  at  the  same  angle  every  time.  To  reduce  the  amount  of 
variability  in  this  experiment  the  Weaver  firing  positions  (standing  and  kneeling)  were 
not  allowed. 

The  isosceles  high  kneeling  position  was  an  attempt  to  ensure  that  every 
participant  will  assume  the  same  position  every  time.  A  kneeling  position  was  more 
important  because  the  author  wanted  to  make  sure  the  participants  were  implementing 
their  marksmanship  skills  that  they  acquired  from  the  marksmanship  stimulus  training. 
The  pistol  is  constantly  moving  and  the  participants  had  to  focus  on  their  breathing,  sight 
picture  and  sight  alignment  along  with  the  other  fundamentals.  In  the  Weaver  high 
kneeling  position,  the  participants  could  sit  on  their  foot  and  rest  their  elbow  on  their 
knee  to  try  and  stabilize  their  body. 

By  forcing  every  participant  to  assume  the  isosceles  high  kneeling  position,  the 
author  reduced  the  amount  of  variability  in  marksmanship  performance  that  could  be 
accounted  for  by  differences  in  position.  This  position  forced  the  participants  to  focus  on 
all  the  elements  of  marksmanship  fundamentals  similar  to  the  standing  position.  If  the 
participant  failed  to  maintain  proper  muscular  tension  or  properly  place  the  finger  on  the 
trigger  then  the  performance  should  vary  more  in  the  isosceles  high  kneeling  position  as 
opposed  the  being  supported  from  the  leading  knee  and  sitting  on  the  back  foot  with  the 
weaver  high  kneeling  position. 

Every  participant  stayed  behind  the  firing  line  during  training  and  qualifications. 

They  stayed  behind  the  ready  line  until  instructed  to  move  forward  to  the  firing  line  by 

the  instructor.  A  blue  sticker  was  placed  as  a  guide  to  help  all  participants  center 
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themselves  on  the  target  at  all  times.  Every  partieipant  used  the  same  holster,  which  was 
connected  to  the  security  belt,  and  the  pistol  rested  on  the  side  of  the  hip  next  to  the 
participants  shooting  hand.  If  the  security  belt  did  not  fit  a  participant,  the  instructor 
detached  the  same  holster  and  attached  it  to  the  participant’s  personal  belt.  The  author 
assumed  that  the  difference  in  belt  type  would  not  have  a  significant  effect  on  the  study, 
but  made  sure  that  every  participant  used  the  same  style  of  holster  to  draw  from. 

The  participants  ensured  that  before  agreeing  to  the  day  and  time  of  their  second 
qualification  that  the  same  day  and  time  was  available  for  the  following  week.  The 
procedure  ensured  that  one  group  did  not  have  an  unfair  advantage  over  the  other  simply 
because  someone  waited  longer  to  complete  their  retention  (third)  qualification. 
Marksmanship  is  a  perishable  skill  and  the  purpose  of  the  retention  phase  was  to 
determine  if  participants  would  retain  what  they  had  learned.  It  would  not  be  fair  if  some 
participants  waited  two  weeks  when  everyone  else  had  one. 

B.  PARTICIPANTS 

1.  Establishing  Control  or  Experimental  Group 

Participants  were  recruited  by  email  or  in  person  to  volunteer  for  the  experiment. 
The  participants  received  the  recruitment  email  with  the  demographic  survey  attached. 
They  were  instructed  to  fill  out  the  demographic  survey  and  submit  it  prior  to  their  first 
qualification.  After  the  participants  completed  their  baseline  qualification,  the  instructor 
reviewed  their  score  and  demographic  survey  and  ranked  the  participant  from  highest  to 
lowest.  Participants  were  then  blocked  by  this  rank  into  the  control  and  experimental 
groups.  This  process  attempted  to  ensure  that  each  group  had  similar  distributions  of 
marksmanship  ability.  It  is  important  to  note  that  the  focus  of  the  experiment  was  not  the 
scores  after  completion  of  each  qualification,  but  the  difference  between  the  scores  on  all 
performance  measures  to  show  any  changes.  For  example,  if  a  participant  started  out  as 
an  expert  on  the  baseline  (first)  qualification,  changes  in  his/her  performance  could  still 
be  detected  by  the  individual  performance  measures,  such  as  shot  groups  and  MPI  to  CZP 
lengths. 
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2.  Demographic  Statistics 

A  demographic  survey  was  administered  to  every  participant  at  the  beginning  of 
the  experiment  to  try  and  get  an  understanding  of  their  level  of  marksmanship.  This 
information  along  with  their  baseline  score  was  required  in  order  to  assign  them  to  the 
control  or  experimental  group.  Refer  to  Table  5  for  descriptive  statistics  on  participants’ 
demographic  characteristics.  Refer  to  Appendix  H  for  a  copy  of  the  demographic  survey. 


Demographic  Survey 

Control  Group 

Experimental  Group 

Age  18-25 

0.07 

0.13 

Age  25-30 

0.27 

0.27 

Age  31-35 

0.40 

0.33 

Age  36-40 

0.20 

0.27 

Age  41+ 

0.07 

0.00 

Male 

0.87 

0.80 

Right  Handed 

0.S3 

0.93 

Wear  Corrective  Lens 

0.13 

0.27 

Civilian 

0.07 

0.27 

Military 

0.93 

0.73 

Navy 

0.47 

0.40 

Army 

0.20 

0.20 

Marine 

0.13 

0.00 

Air  Force 

0.07 

0.07 

Inter  Student 

0.07 

0.07 

Year  of  Service  {Mean} 

11.63 

8.47 

Year  of  Service  {SD} 

5.43 

6.51 

Marksmanship 

Experience 

Novice 

0.27 

0.33 

Marksman 

0.33 

0.40 

Sharpshooter 

0.33 

0.07 

Expert 

0.07 

0.20 

Table  5.  Descriptive  Statistics  on  Participants’  Demographic  Characteristics 

The  demographic  survey  revealed  that  there  were  no  significant  differences 
between  the  control  and  experimental  group  based  on  age  group,  gender  or  dominant 
hand.  The  experimental  group  had  more  participants  that  wore  corrective  lens  than  the 
control  group.  The  author  ensured  if  a  participant  started  the  experiment  with  or  without 
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their  corrective  lens  that  they  consistently  used/didn’t  use  their  corrective  lens  through 
the  entire  experiment.  The  experimental  group  also  had  a  few  more  civilians  and  self- 
reported  novices,  but  the  control  group  had  fewer  self-reported  experts.  It  is  also 
important  to  show  that  the  percentages  of  participants  in  separate  branch  of  services  are 
close,  but  the  control  group  had  more  years  in  service.  In  summary,  the  two  groups 
differed  only  in  years  of  service  and  in  the  number  of  self-reported  sharpshooters  and 
novice. 


3.  Participant  Agenda 

The  purpose  of  this  section  is  to  provide  description  of  the  scheduling 
requirements  expected  from  every  participant.  The  schedule  also  explains  the  chain  of 
events  from  a  day-to-day  projection.  Refer  to  Table  6  for  the  participant  schedule. 


Week  1 

Monday 

Tuesday 

Wednesday 

Thursday 

Control  Group 
Baseline  Qual 

Control  Group 
Training  Qual 

Simulation  Group 
Baseline  Qual 

Simulation  Group 
Training  Qual 

Dayl 

Day2 

Day  3 

Day  4 

Week  2 

Monday 

Tuesday 

Wednesday 

Thursday 

Control  Group 
Retention  Qual 

Simulation  Group 
Retention  Qual 

Day9 

Dayll 

Table  6.  Participant  schedule 


(Day  0)  A  demographic  survey  was  administered  to  all  participants  prior  to  the 
beginning  of  the  experiment  so  as  to  be  able  to  block  participants  into  the  control  and 
experimental  groups  by  previous  marksmanship  experience.  It  contained  questions  that 
asked  about  the  participants’  characteristics,  marksmanship  skill  level  and  experience. 

(Day  1)  The  ISMT  qualification  on  the  standard  navy  handgun  qualification  COF 
was  administered  to  the  control  group  to  establish  a  baseline  of  their  current  level  of 
marksmanship  performance  and  score.  Their  baseline  qualification  score  and  the 
information  on  the  demographic  survey  from  day  0  were  used  to  block  participants  into 
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the  control  and  experimental  groups.  The  standard  navy  handgun  qualification  COF 
procedure  was  checked  by  a  subject  matter  expert  (SME)  in  order  to  ensure  that  the 
proper  instruction  was  given. 

(Day  2)  The  control  group  received  current  naval  marksmanship  training,  which 
involves  handling  a  simulated  9mm  Berretta,  practicing  proper  techniques  for  standing, 
breathing,  and  doing  dry  fire.  ISMT  qualification  on  the  standard  Navy  handgun 
qualification  COF  was  administered  to  determine  the  main  effects  of  the  training  on  the 
control  group’s  marksmanship  performance  and  score.  The  current  naval  marksmanship 
training  was  checked  by  a  SME  in  order  to  ensure  that  the  proper  training  was  given. 

(Day  3)  The  ISMT  qualification  on  the  standard  navy  handgun  qualification  COF 
was  administered  to  the  experimental  group  to  establish  a  baseline  of  their  current  level 
of  marksmanship  performance  and  score.  Their  baseline  qualification  score  and  the 
information  on  the  demographic  survey  from  day  0  were  used  to  block  participants  into 
the  control  and  experimental  groups.  The  standard  navy  handgun  qualification  COF 
procedure  was  checked  by  a  SME  in  order  to  ensure  that  the  proper  instruction  was  given. 

(Day  4)  The  experimental  group  received  simulation-based  marksmanship 
training  in  the  ISMT.  This  training  involves  handling  a  simulated  9mm  berretta,  practice 
proper  techniques  for  standing  and  breathing.  These  participants  also  had  the  opportunity 
to  fire  practice  shots  at  simulated  targets,  analyze  their  trace  profiles  and  shot  groups.  The 
immediate  feedback  provided  by  ISMT  was  the  only  aspect  of  the  training  that  was 
different  from  the  control  group.  ISMT  qualification  on  the  standard  Navy  handgun 
qualification  COF  was  administered  to  determine  the  main  effects  of  the  training  on  the 
experimental  group’s  marksmanship  performance  and  score.  The  simulation-based 
marksmanship  training  was  checked  by  a  SME  in  order  to  ensure  that  the  proper  training 
was  given. 

(Days  5-8)  No  additional  training  was  provided. 

(Day  9)  ISMT  qualification  on  the  standard  navy  handgun  qualification  COF  was 
administered  to  the  control  group.  This  determined  how  much  marksmanship  training  the 
participant  had  retained. 
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(Day  10)  No  additional  training  was  provided. 

(Day  11)  ISMT  qualification  on  the  standard  navy  handgun  qualification  COF 
was  administered  to  the  experimental  (simulation)  group.  This  determined  how  much 
marksmanship  training  the  participant  had  retained. 

During  all  training  and  qualification  assessments,  data  regarding  participants’ 
performance  score,  diameter  of  shot  groups,  and  trace  profiles  were  collected.  After  the 
participants  had  left  the  ISMT  lab,  a  digital  camera  was  used  as  backup  to  collect 
performance  and  score  data  that  was  projected  on  the  screen.  The  participants  were  not 
video  or  audio  recorded  at  any  time. 

C.  EQUIPMENT 

This  section  describes  the  ISMT  and  how  it  works.  The  ISMT  enclosure  holds  the 
computer  processing  unit  (CPU),  amplifier  and  a  flip  up  monitor  attached  to  a  keyboard. 
The  following  components  are  also  a  part  of  the  ISMT  suite;  two  speakers,  projector, 
projector  screen,  hit  camera,  simulated  M9  Berretta  and  a  CO^  tank.  The  CPU 
synchronized  all  the  components  in  order  to  run  the  simulator.  The  ISMT  lab  for  this 
thesis  utilized  an  additional  computer  monitor  and  keyboard  so  that  ISMT  can  be 
controlled  from  multiple  stations.  The  CPU  sent  video  data  to  the  projector  that  projected 
an  image  on  the  screen  mounted  against  the  wall  ahead.  The  participant  fired  a  round  at  a 
target  on  the  screen  with  a  M9  pistol  that  is  connected  to  the  CPU  and  a  carbon  dioxide 
(CO^)  tank.  The  CO^  tank  released  gas  that  was  used  to  simulate  recoil  in  the  pistol.  The 
pistol  sent  an  infrared  beam  out  of  the  barrel  and  that  signal  was  received  by  a  hit  camera 
that  is  connected  to  the  CPU.  The  CPU  sent  an  audio  signal  to  the  amplifier  which  is 
connected  to  two  speakers  located  on  the  floor,  to  the  right  and  left  of  the  screen.  The 
speakers  simulated  the  sound  of  a  M9  pistol  being  fired.  The  CPU  calculated  the  shot 
fired  based  on  the  information  received  from  the  hit  camera  and  presented  the 
marksmanship  performance  data  at  the  end  of  the  qualification.  Refer  to  Figure  6  for  a 
picture  of  the  ISMT  major  components.  The  labels  were  added  to  the  picture  to  assist 
identifying  components. 
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Figure  6.  ISMT  Major  Components  (after  Schwetje,  2009) 


1.  Building  Scenarios 

The  ISMT  used  in  this  experiment  was  the  Marine  Corp  version.  All  the 
preinstalled  training  programs  were  within  the  Marine  Corp  standards.  Therefore,  the 
author  programmed  the  ISMT  to  establish  the  navy  handgun  qualification  standards.  The 
following  paragraphs  will  describe  each  of  the  scenarios  and  its  purpose. 

a.  Zero  Scenario 

The  zero  scenarios  for  each  of  the  three,  seven  and  15-yard  lines  were 
created.  In  real  life,  a  pistol  does  not  get  zeroed  like  a  rifle  because  the  sights  do  not 
move.  But  these  scenarios  ensured  the  accuracy  of  the  simulated  pistol,  thus  the  zero 
scenarios  were  used.  The  zero  scenarios  were  also  used  for  quick  operational  test  of  the 
M9  Berettas,  which  assisted  with  trouble  shooting  replacement  pistols  to  ensure  their 
integrity  during  the  experiment.  All  zero  scenarios  have  no  time  limit  and  have  only  one 
magazine  of  three  rounds.  Refer  to  Appendix  B,  C  and  D  for  screen  captures  of  every 
setting  to  recreate  the  three,  seven  and  15  yard  line  zero  scenarios. 
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b.  Qualification  Scenario 


The  qualification  scenario  was  the  most  important  because  all  the  data 
collection  would  come  from  the  results  of  the  qualification  at  the  end  of  the  scenario. 
After  a  few  trials  the  author  determined  it  was  best  to  have  a  target  to  represent  the 
distance  of  three,  seven  and  15-yard  line.  The  targets  would  not  appear  until  the  recorded 
COF  introduction  instructions  were  completed.  Then  the  targets  appeared  prior  to  the  first 
shot  of  that  particular  stage  of  fire.  The  targets  disappeared  after  the  last  cease-fire 
instruction  was  completed  per  stage.  This  allowed  the  participants  to  focus  on  the 
instructions  versus  the  target.  Refer  to  Table  7  for  a  complete  list  of  all  the  stages  and 
required  sequence  of  fire. 


Navy  Handgun  Qualification  Course 

Yard  Line 

Sequence 

Time  Limit 

Remarks 

3  Yards 

Draw  and  fire  2  rounds 

4  seconds 

Strong  Hand  Supported 

3  Yards 

Draw  and  fire  2  rounds 

4  seconds 

Strong  Hand  Supported 

3  Yards 

Draw  and  fire  2  rounds,  reload  and  fire  2  rounds 

10  seconds 

Strong  Hand  Supported 

3  Yards 

Draw  and  fire  4  rounds 

8  seconds 

2  rounds  Strong  Hand,  2  rounds  weak  Hand  supported 

7  Yards 

Draw  and  fire  2  rounds 

4  seconds 

Strong  Hand  Supported 

7  Yards 

Draw  and  fire  2  rounds 

4  seconds 

Strong  Hand  Supported 

7  Yards 

Draw  and  fire  2  rounds,  reload  and  fire  2  rounds 

10  seconds 

Strong  Hand  Supported 

7  Yards 

Draw  and  fire  4  rounds 

8  seconds 

2  rounds  Strong  Hand,  2  rounds  weak  Hand  supported 

15  Yards 

Draw  and  fire  2  rounds 

4  seconds 

Strong  Hand  Supported 

15  Yards 

Draw  and  fire  2  rounds 

4  seconds 

Strong  Hand  Supported 

15  Yards 

Draw  and  fire  4  rounds 

8  seconds 

Strong  Hand  Supported 

15  Yards 

Draw  and  fire  4  rounds,  reload  and  fire  4  rounds 

20  seconds 

Strong  Hand  Supported 

15  Yards 

Draw  and  fire  8  rounds 

20  seconds 

Strong  Hand  Supported 

Table  7.  Navy  Handgun  Qualification  COF 


The  qualification  scenarios  provided  the  participant  with  two  magazines 
(six  rounds  in  each  magazine)  for  the  three  and  seven  yard  line  COF.  The  15-yard  line 
had  two  magazines  with  12  rounds  in  each  magazine.  The  recorded  navy  handgun 
qualification  COF  commands  were  used  to  administer  the  qualification.  The  scene 
selected  for  this  scenario  was  a  picture  the  author  took  of  a  flight  deck  looking  out  at  sea. 
The  image  was  imported  into  ISMT  and  the  intention  was  to  help  the  participant  develop 
a  navy  centric  experience  of  a  handgun  qualification  out  at  sea.  Refer  to  Appendix  E  for 
screen  captures  of  every  setting  in  the  qualification  scenarios. 


33 


c. 


Unlimited  Practice  Scenario 


The  unlimited  practice  scenario  was  created  so  that  all  participants  could 
practice  with  an  unlimited  amount  of  time.  The  participants  were  presented  with  a  three, 
seven  and  15 -yard  line  targets  on  the  screen  to  take  practice  shots.  The  control  group 
utilized  this  scenario  to  conduct  dry  fire  training  and  practice  taking  aim  at  the  targets. 
The  pistol  was  disconnected  from  the  ISMT  and  the  CO^  tank.  This  procedure  ensured 
that  the  participants  in  the  control  group  did  not  receive  any  immediate  feedback  from  the 
ISMT.  The  control  group  only  received  immediate  feedback  from  the  instructor.  The 
instructor  paid  close  attention  so  that  a  pistol  is  connected  or  disconnected  based  on  the 
training  phase  or  if  the  participant  was  in  the  control  group  versus  the  experimental 
group. 

This  scenario  provided  an  unlimited  supply  of  magazines  to  the 
experimental  group.  ISMT  does  not  account  for  an  unlimited  supply  of  magazine  but 
provided  the  maximum  allotted  magazines  with  six  rounds  in  each  magazine.  The  author 
decided  that  in  order  to  maximize  training,  the  magazines  should  have  six  rounds  instead 
of  12  rounds  because  it  forced  the  participants  in  the  experimental  group  to  reload  the 
pistol  frequently.  The  scenario  provided  instant  feedback  to  the  participant  in  the 
experimental  group.  Experimental  group  participants  were  able  to  view  a  silhouette  of  the 
targets  in  a  blue  box  at  the  top  of  the  projection  screen.  If  the  participants  in  the 
experimental  group  had  a  hard  time  seeing  a  target  silhouette  to  track  the  position  of  the 
shots  fired,  they  were  allowed  to  look  at  the  instructor’s  computer  monitor.  The  monitor 
displayed  the  same  information  but  with  a  zoomed  in  view. 

All  the  participants  were  asked  to  square  their  shoulders  to  the  target  by 
shuffling  to  the  right  or  left.  This  adjustment  ensured  that  the  targets  were  centered  in 
front  of  the  participant  when  they  practiced  firing  the  pistol.  The  author  did  not  want  to 
risk  any  negative  training  with  the  participants  by  allowing  them  to  stand  in  one  place 
and  shoot  at  a  target  offset  or  not  directly  in  front  of  the  shooter.  If  the  instructor  allowed 
the  participant  to  aim  at  a  target  off  to  the  side  rather  than  directly  in  front,  the  participant 
looked  at  an  improper  sight  picture  and  sight  alignment.  In  order  to  ensure  the  continuity 

of  good  sight  picture  and  sight  alignment,  the  participants  needed  to  see  the  same  thing 
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every  time.  Refer  to  Appendix  F  for  screen  captures  of  every  setting  to  recreate  the 
unlimited  practice  scenario. 

d.  Practice  Qualification  Scenario 

The  practice  qualification  scenario  was  almost  identical  to  the 
qualification  scenario  except  there  were  pauses  between  the  three,  seven  and  15-yard  line. 
This  scenario  had  two  more  sub  serials  than  the  qualification  scenario.  Refer  to  the  ISMT 
manual  and  Appendix  G  for  information  on  sub  serials.  The  scenario  provided  a  view  of 
the  target's  silhouette  in  a  blue  box  for  the  experimental  group.  These  targets  were  located 
at  the  top  of  the  projection  screen  the  above  the  actual  target  that  the  participants  fired  at. 
If  the  participants  had  a  hard  time  seeing  the  silhouette  to  keep  track  of  the  position  of  the 
shots  fired,  they  were  allowed  to  look  at  the  instructor’s  computer  monitor  that  displays 
the  same  information  but  with  a  zoomed  in  view.  The  control  group  also  used  this 
scenario  but  the  pistol  was  disconnected  from  the  ISMT  and  the  CO  tank.  The  instructor 
paid  close  attention  so  that  a  pistol  is  connected  or  disconnected  based  on  the  training 
phase  or  if  the  participant  was  in  the  control  group  versus  the  experimental  group.  The 
same  amount  of  magazines  and  rounds  in  the  qualification  scenario  applied  to  the 
practice  qualification.  The  recorded  navy  handgun  qualification  COF  commands  were 
used  to  administer  the  qualification.  Refer  to  Appendix  G  for  screen  captures  of  every 
setting  of  the  practice  qualification  scenario. 

2.  Voice  Recording  Instructions 

The  author  decided  that  it  was  imperative  the  participant  received  their  basic 
training,  control  group  stimulus  training  or  experimental  group  stimulus  training,  and 
instructions  for  the  standard  navy  handgun  qualification  COF  from  a  recording.  The 
author  planned  on  having  30  participants  and  each  participant  had  to  qualify  three  times 
each.  He  felt  that  he  would  not  be  able  to  deliver  the  instructions  and  training  the  exact 
same  way  throughout  the  entire  data  collection  period.  Special  care  was  required  in  the 
creation  of  the  media  files  and  timing  the  ISMT  scenarios  so  that  the  targets  appeared  and 
disappeared  in  a  timely  manner. 
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3.  Lab  Setup 

The  ready  line  and  firing  line  was  established  by  laying  duct  tape  for  the  firing 
line  and  masking  tape  for  the  ready  line.  The  instructor  also  posted  a  copy  of  the  four 
general  safety  rules  so  that  participants  viewed  and  referenced  them  at  all  times.  It  was 
important  to  have  a  separate  laptop  connected  to  speakers  so  that  the  instructor  played  the 
playlists  independently  of  the  ISMT.  Before  the  experiment  began,  special  care  was  taken 
with  ISMT  preparations  by  calibrating  the  hit  camera  after  the  system  startup.  After  the 
system  was  up  and  running  and  the  system  calibration  was  complete  then  the  M9  Berretta 
went  through  a  weapon  registration  and  weapon  calibration.  At  this  point  the  instructor 
began  to  collect  data.  Every  participant  went  through  a  shooter  registration  in  ISMT  to 
help  keep  track  of  the  participants’  four-digit  identification. 

4.  Data  Recording 

Based  on  lessons  learned  in  a  class  project,  the  author  decided  it  was  best  to  have 
several  backup  systems  for  data  collections.  Pictures  were  taken  at  the  completion  of 
every  qualification.  The  instructor  took  pictures  of  the  results  page,  all  the  shot  groups, 
all  the  MPI  to  CZP  groups  and  all  48  trace  profiles  for  every  shot  fired  in  the 
qualification.  An  Excel  spreadsheet  also  was  used  to  collect  data  and  conduct  statistical 
analysis.  The  instructor  printed  out  the  results  data  after  the  pictures  were  taken  and  the 
data  was  transferred  to  Excel.  The  print  outs  were  filed  in  case  any  electronic  files 
became  corrupted  or  lost. 

D.  MAIN  MEASURES 

The  main  measures  for  this  experiment  were  the  various  marksmanship 
performance  data  that  ISMT  provided  at  the  end  of  each  qualification  and  a  post 
experiment  survey.  This  section  lists  each  of  the  performance  measures  used.  It  also 
provides  a  description  of  how  they  were  scored  and  indicates  whether  higher  or  lower 
scores  were  better. 
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1.  Scores  on  the  Standard  Navy  Handgun  Qualification  COF 

These  scores  are  based  on  points  earned  depending  on  which  region  of  the  target 
was  hit.  The  more  points  a  participant  received  for  a  qualification,  the  better. 
Marksmanship  improvement  with  this  measure  was  determined  when  participants 
received  higher  scores  on  training  (second)  qualification  than  the  baseline  (first) 
qualification.  The  same  determination  was  made  when  a  participant  received  higher 
scores  on  the  retention  (third)  qualification  than  the  training  (second)  qualification.  Thus, 
positive  difference  scores  (e.g.,  training  scores — ^baseline  scores)  indicate  improvement. 
Transtar  II  targets  were  used  for  this  experiment  and  were  in  accordance  with  small  arms 
training  and  qualificaiton  instruction,  enclosure  seven,  section  lA  (Chief  of  Naval 
Operations,  2009).  The  target  was  split  up  into  six  regions.  Region  one  was  worth  zero 
points,  region  two  was  worth  two  points,  region  three  was  worth  three  points,  region  four 
was  worth  four  points,  region  five  and  six  were  worth  five  points.  Refer  to  Figure  7  for  a 
picture  of  a  transtar  II  target.  The  image  also  identifies  the  regions  of  the  target. 
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Region  1  Outside  of  Target  Silhouette 


Figure  7.  Transtar  II  Target  with  Region  Labels 

2.  Diameter  Size  of  the  Shot  Group 

Marksmanship  performance  was  measured  by  the  diameter  size  of  the  three,  seven 
and  15-yard  line  shot  group.  Diameter  size  measures  variability  in  shot  consistency.  All 
shot  group  size  diameters  were  measured  in  centimeters  and  a  participant  showed 
improvement  when  the  diameter  decreased  over  time.  Therefore,  negative  difference 
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scores  (training  scores  -  baseline  scores)  indicate  improvement  from  baseline  to  training. 
Special  consideration  determined  it  was  best  to  only  use  three  targets  that  represented  the 
three,  seven  and  15-yard  line.  The  scenarios  could  not  use  a  new  target  for  every  step 
during  the  COF  because  ISMT  cannot  determine  a  shot  group  diameter  with  only  two 
rounds.  Therefore,  the  three  and  seven  yard  line  targets  received  12  rounds  each,  while 
the  15 -yard  line  target  received  24  rounds. 

3.  Average  Shot  Group 

Average  diameter  size  measures  variability  in  shot  consistency.  All  shot  group 
size  diameters  were  measured  in  centimeters  and  a  participant  showed  improvement 
when  the  diameter  decreased  over  time.  Therefore,  negative  difference  scores  (training 
scores  -  baseline  scores)  indicate  improvement  from  baseline  to  training.  Thus,  as  in  the 
diameter  size  of  shot  groups,  negative  difference  scores  in  the  average  shot  group  indicate 
improvement. 

4.  MPI  to  CZP 

The  length  of  mean  point  of  impact  to  center  zeroing  point  (MPI  to  CZP) 
measures  the  average  distance  of  the  shot  fired  to  a  fixed  point  on  the  target.  The  fixed 
point  on  the  target  was  determined  by  the  author  when  he  placed  it  in  the  center  of  region 
six,  during  the  creation  of  all  the  targets.  MPI  to  CZP  was  measured  for  the  three,  seven 
and  15-yard  line.  The  length  of  MPI  to  CZP  also  was  measured  in  centimeters  and  a 
participant  showed  improvement  with  accuracy  when  the  length  decreased  over  time. 

5.  Demographic  Survey 

The  demographic  survey  was  administered  in  order  to  block  participants  into  the 
control  and  experimental  groups.  It  contained  questions  that  asked  about  the  participants’ 
characteristics,  marksmanship  skill  level  and  experience. 

6.  Post  Experiment  Survey 

After  completing  the  experiment,  participants  were  asked  four  open  ended 
questions  to  determine  their  thoughts  about  the  experiment,  how  they  felt  about 
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simulators  and  if  they  were  willing  to  replace  real  marksmanship  practice  with  simulated 
marksmanship  practice.  Refer  to  Appendix  1  for  a  copy  of  the  post  survey.  The  data 
cannot  determine  if  an  improvement  occurred  in  the  participant’s  opinion  because  a 
preliminary  open-ended  survey  was  not  administered. 
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III.  RESULTS 


A.  DATA  PREPARATION  AND  PRELIMINARY  ANALYSES 

This  chapter  will  report  results  from  hypothesis  testing  for  this  thesis.  The  data 
was  recorded  using  Excel  and  all  statistical  analysis  was  conducted  using  the  data 
analysis  feature  for  Excel.  This  section  describes  data  preparation  and  preliminary 
analyses. 

1.  Data  Preparation 

The  data  preparation  for  this  experiment  consisted  of  calculating  the  difference 
scores  on  all  the  main  measures.  Three  sets  of  difference  scores  were  calculated:  (1)  the 
difference  between  baseline  and  post  training  performance  (baseline  -  post  training)  to 
test  hypothesis  one;  (2)  the  difference  between  post  training  and  post  retention  scores 
(post  training — post  retention)  were  calculated  to  test  hypothesis  3;  (3)  the  difference 
between  baseline  and  post  retention  scores  (baseline  -  post  retention)  also  were  calculated 
to  explore  whether  group  differences  would  be  stronger  after  a  longer  time  frame.  Two 
sample  Mests  of  these  difference  scores  were  used  to  determine  if  there  were  group 
differences  in  improvement.  Prior  to  performing  the  two  sample  t-tests,  F-tests  were  used 
to  check  equal  variance  assumption  for  all  comparisons.  When  the  F-test  indicated  that 
the  equal  variance  assumption  was  met,  the  two-sample  t-test  assuming  equal  variances 
was  used;  otherwise,  the  two  sample  Mest  assuming  unequal  variances  was  used.  The 
two-sample  t-test  assuming  equal  variances  was  used  for  all  main  measure  except  two. 
The  three-yard  line  score  and  the  three-yard  line  MPI  TO  CZP  did  not  meet  the  equal 
variance  assumption  between  the  retention  (third)  qualification  and  the  training  (second) 
qualification.  For  these  two  cases,  the  two-sample  t-test  assuming  unequal  variance  was 
used.  The  normality  assumption  also  was  checked.  Non-parametric  tests  (Wilcoxin  Rank 
Sum  Test)  were  used  when  the  normality  assumption  was  violated.  The  results  from  all 
non-parametric  tests  had  the  same  pattern  as  two  sample  t-tests.  Therefore,  only  two 
sample  t-test  results  are  described.  For  all  tests,  an  alpha  of  .05  one  tailed  was  used. 
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2.  Preliminary  Analyses 

Before  conducting  hypothesis  testing,  the  author  first  wanted  to  check  that  the 
control  and  experimental  groups  performed  similarly  at  baseline.  A  two  sample  t-test 
assuming  equal  variance  was  conducted  on  the  baseline  final  qualification  scores  (0-240 
possible  points)  between  the  control  group  and  experimental  group.  Results  indicate  that 
there  is  no  significant  difference  between  the  control  and  experimental  group  in  baseline 
performance  (see  Table  8).  Therefore,  even  though  there  was  a  group  difference  in  the 
distribution  of  self-reported  marksmanship  experience,  this  result  indicates  that  on  an 
objective  measure  of  performance,  the  groups  were  evenly  matched  in  ability. 

The  next  section  will  take  a  closer  look  at  marksmanship  performance. 


t-Test:  Two-Sample  Assuming  Equal  Variances 


Baseline  Scores  =  Week  1 

Control 

Sim 

Mean 

2Q3A 

201.S666667 

Variance 

422.Z57142S 

9S1.409523S 

Observations 

15 

15 

Pooled  Variance 

TO1.S333333 

Hypothesized  Mean  Difference 

0 

df 

2S 

tStat 

0.77S754S51 

P{T<=t:)  one-tail 

t  Critical  one-tail 

1.7O1130S0S 

P{T<^)  t\'vo-tail 

t  Critical  two-tail 

2.04S407115 

Table  8.  Baseline  Scores  Two  Sample  Assuming  Equal  Variance  t-Test 


This  section  will  provide  the  means  and  standard  deviations  for  the  baseline, 
training  and  the  retention  phase  of  the  experiment.  The  purpose  for  reviewing  the 
descriptive  statistics  during  each  phase  was  to  gain  some  preliminary  determination  of  the 
progress  on  each  group’s  performance.  Therefore,  formal  statistical  testing  was  not 
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conducted  in  order  to  keep  type  II  error  low.  Refer  to  Table  9  for  the  deseriptive  statisties 
during  the  baseline  phase. 


Descriptive  Statistics 


Baseline 

Scores 

Ave  Shot 

Group  Size 

3  Yard  Shot 

Group  Size 

3  Yard  MPI 

to  CZP 

7  Yard  Shot 

Group  Size 

7  Yard  MPI 

to  CZP 

15  Yard 
Shot  Group 

Size 

15  Yard 

MPI  to  CZP 

Control  Group  Mean 

203. m 

3.77 

10.03 

2.30 

18.81 

5.81 

50.61 

8.76 

Control  GroupSD 

20.55 

4.20 

6.43 

1.15 

6.46 

3.28 

27.77 

5.18 

Ej«perinnental  Group  Mean 

201.87 

12.02 

13.57 

2.81 

27.18 

7.45 

54.32 

8.73 

Experinnental  Group  SD 

31.33 

5.32 

8.35 

1.76 

15.10 

4.01 

32.50 

4.01 

Table  9.  Baseline  Descriptive  Statisties 


All  of  the  data  colleeted  are  measured  in  centimeters  with  the  exeeption  of  the 
scores.  The  baseline  deseriptive  statistics  suggested  that  the  eontrol  group  was 
performing  slightly  better  than  the  experimental  group,  but  note  the  large  standard 
deviations.  The  eontrol  group  has  a  higher  seore  and  appears  to  have  a  smaller  shot 
group.  The  experimental  group  appears  to  be  slightly  more  accurate  on  the  three-  and  15- 
yard  line  based  on  the  MPI  to  CZP.  Refer  to  Table  10  for  the  deseriptive  statisties  during 
the  training  phase. 


Descriptive  Statistics 


Training 

Scores 

Ave  Shot 

Group  Size 

3  Yard  Shot 

Group  Size 

3  Yard  MPI 

to  CZP 

7  Yard  Shot 

Group  Size 

7Yard  MPI 

to  CZP 

15  Yard 
Shot  Group 

Size 

15  Yard 

MPI  to  CZP 

Control  Group  Mean 

227.60 

6.75 

6.74 

1.81 

15.12 

4.13 

35.37 

7.18 

Control  GroupSD 

7.26 

1.82 

2.25 

1.08 

5.52 

1.73 

12.37 

2.88 

Experimental  Group  Mean 

225.53 

7.11 

7.81 

1.57 

15.83 

2.88 

33.23 

5.50 

Experimental  GroupSD 

18.87 

2.74 

3.28 

1.00 

5.73 

2.86 

16.26 

3.48 

Table  10.  Training  Descriptive  Statistics 

The  training  deseriptive  statisties  suggest  that  the  control  group  and  the 
simulation  group  performed  at  similar  levels.  The  deseriptive  statisties  suggest  that  as  the 
distance  from  the  target  inereased,  the  experimental  group  decreased  their  shot  group 
size.  The  eontrol  group  did  not  appear  to  be  as  aecurate  as  the  experimental  group  based 
on  the  length  of  the  MPI  to  CZP  on  the  three-,  seven-  or  15-yard  line.  The  experimental 
group  had  more  variability  on  the  seores,  indicating  wider  range  in  partieipant 
performance.  Refer  to  Table  1 1  for  the  deseriptive  statistics  during  the  retention  phase. 
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Descriptive  Statistics 


Retention 

Scores 

Ave  Shot 

Group  Size 

3  Yard  Shot 

Group  Size 

3  Yard  MPI 

to  CZP 

7  Yard  Shot 

Group  Size 

7Yard  MPI 

to  CZP 

15  Yard 
Shot  Group 

Size 

15  Yard 

MPI  to  CZP 

Control  Group  Mean 

222.07 

7.21 

7.84 

2.41 

16.4S 

5.S7 

33.48 

7.63 

Control  Group  SD 

IS.  14 

2.S6 

3.82 

1.21 

8.38 

3.3S 

14.71 

3. SO 

E>:perinnental  Group  Mean 

223.53 

7.41 

8.6S 

2.28 

15.66 

4.33 

34.45 

8.18 

Experinnental  Group  SD 

17.63 

2.56 

3. SI 

1.71 

5.72 

2.5S 

3.52 

4.71 

Table  11.  Retention  Deseriptive  Statistics 


The  retention  descriptive  statistics  suggest  that  the  control  group  and  the 
simulation  group  have  similar  marksmanship  performance  levels.  The  measurements  for 
the  shot  groups  and  MPI  to  CZP  are  very  close  but  the  control  group  has  a  larger  standard 
deviation  for  the  15-yard  line  shot  group  size. 


3.  Results  from  Hypothesis  Testing 

This  section  addressed  results  from  each  of  the  three  hypotheses. 


a.  Hypothesis  One 

HO:  There  will  be  no  change  in  the  participants’  marksmanship  performance  and 
scores  in  the  Standard  Navy  Handgun  Qualification  Course  of  fire  (OPNAVINST 
3591.  IF,  2009)  from  the  baseline  performance  to  training  phase. 

HA:  Participants  who  receive  simulation-based  training  in  ISMT  will  have  a 
greater  improvement  in  marksmanship  performance  and  scores  in  the  standard  Navy 
Handgun  Qualification  Course  of  fire  (OPNAVINST  3591.  IF,  2009)  from  the  baseline 
performance  to  the  training  phases  than  participants  who  received  current  naval 
marksmanship  training. 

There  were  no  significant  differences  in  improvement  between  the  control  and  the 
experimental  group  for  the  average  shot  group  size,  three-yard  line  shot  group  size,  three 
yard  line  MPI  to  CZP,  15-yard  line  shot  group  size  and  15-yard  line  MPI  to  CZP. 
Therefore,  the  null  hypothesis  was  retained  for  these  variables. 

There  were  significant  differences  in  the  seven  yard  line  shot  group  size  {p 
=  .007)  and  seven  yard  line  MPI  to  CZP  {p  =  .048).  Refer  to  Tables  12  and  13  for  the 
results  of  the  seven-yard  line  shot  group  size  and  seven-yard  line  MPI  to  CZP.  A  negative 
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score  indicates  more  accurate  shooting.  For  both  variables,  the  experimental  group 
improved  more  than  the  control  group. 


t-Test:  Two-Sample  Assuming  Equal  Variances 


7  Yard  Line  Training-Baseline  Group  Size 


Con  fro  J 


Sim 


Mean 

Variance 

Observations 

Pooled  Variance 

Hypothesized  Mean  Difference 

df 

tStat 

P{T<=t)  one-tail 
t  Critical  one-tail 
P{T<=t)  two-tail 
t  Critical  two-tail 


-4.7S3333333  -11.25333333 
63  .S 17S095  2  146 .0  24095  2 


15 

104.9209524 

0 

2S 

1. 72715 7S 77 


0.0475  7S559 


1.70113090S 


2.048407115 


15 


SIG  DIFF  between  Control  and  Sim  Group  with  P(T<=t)  one-tail 


Table  12.  Hypothesis  One,  Seven  Yard  Line  Shot  Group  Statistics 


t-Test:  Two-Sample  Assuming  Equal  Variances 

7  Yard  Line  Training-Baseline  MPI  to  CZP 

Control 

s/m 

Mean 

-1.68 

-4.553333333 

Variance 

9.991714286 

7.724095238 

Observations 

15 

15 

Pooled  Variance 

8.857904762 

Hypothesized  Mean  Difference 

0 

df 

28 

tStat 

2.643937225 

P{T<=t}  one-tail 

0.006637884 

t  Critical  one-tail 

1.701130908 

P{T<=t}  two-tail 

0.013275767 

t  Critical  two-tail 

2.048407115 

SIG  DIFF  between  Control  and  Sim  Group  with  P(T<^)  one-tail 

SIG  DIFF  between  Control  and  Sim  Group  with  P(T<^)  two-tail 

Table  13.  Hypothesis  One,  Seven  Yard  Line  MPI  to  CZP  Statistics 
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b.  Hypothesis  Two 

HO:  Among  those  who  do  not  meet  the  qualification  standard  in  the  standard 
Navy  Handgun  Qualification  Course  of  fire  (Chief  of  Naval  Operations,  2009)  at 
baseline,  there  will  be  an  equal  number  of  participants  in  each  group  who  do  not  meet  the 
qualification  standard  after  training. 

HA:  Among  those  who  do  not  meet  the  qualification  standard  in  the  standard 
Navy  Handgun  Qualification  Course  of  fire  (Chief  of  Naval  Operations,  2009)  at 
baseline,  a  greater  number  of  participants  in  the  experimental  group  will  reach 
qualification  standard  after  simulation-based  training  than  in  the  control  group. 

There  were  only  three  people  in  each  group  that  failed  during  the  baseline 
qualification.  Three  per  group  is  insufficient  to  conduct  a  statistical  test  of  any 
significance.  Therefore,  only  descriptive  statistics  are  provided.  Refer  to  Table  14  for  a 
percentage  failures  during  the  entire  experiment. 


Failure  Stats 

Base 

Training 

Final 

Percent  of  Failures  for  Control 

0.13 

0.00 

0.07 

Percent  of  Failures  for  Sim 

0.13 

0.07 

0.07 

Table  14.  Percentage  of  Failures 


A  closer  look  at  scores  for  all  the  failed  participants  using  descriptive  statistics 
suggests  that  this  hypothesis  should  be  tested  in  a  future  study  with  appropriate  sample 
size.  Refer  to  Figure  8  for  a  graphical  depiction  of  the  mean  differences  in  score  during 
the  training  and  retention  qualification  among  those  participants  who  failed  at  baseline. 
Positive  values  indicate  improvement  in  performance. 
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Mean  differences  in  score  by  group 
and  week 


=  40 


Ej 

e 

E 


0 


i 


Control  training  Control  retention  Simtraining  Sim  retention 


Figure  8.  Hypothesis  Two  Mean  Difference  in  Scores  by  Group  and  Week 

The  participants  that  failed  in  the  experimental  group  performed  higher  than  the 
control  group  both  after  training  and  after  retention.  These  descriptive  statistics  suggest 
that  the  experimental  group  enhanced  their  marksmanship  skill  to  a  higher  degree  with 
simulation-based  marksmanship  training  program.  It  also  suggests  that  they  were  able  to 
retain  more  knowledge  and  ability  over  a  one-week  period  of  time  with  no  training. 

To  explore  if  improvement  among  participant  who  failed  at  baseline  was  focused 
on  particular  performance  measures,  descriptive  statistics  on  the  individual  performance 
measures  were  examined.  These  exploratory  analyses  indicated  that  the  greater 
improvement  in  the  experimental  group  is  seen  in  the  seven-yard  line  measures.  Refer  to 
Figure  9  for  a  bar  graph  of  the  seven-yard  line  improvement  from  the  baseline 
qualification  to  the  training  qualification. 
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7  yd  Shot  Group 


7  yd  MPItoCZP 


Figure  9.  Hypothesis  Two  Mean  Difference  in  Seven-yard  Line  Performance  from 

baseline  to  post  training. 


The  descriptive  statistics  did  not  indicate  that  the  participants  that  failed  in  the 
experimental  group  were  able  to  retain  more  marksmanship  skills  then  the  participants 
that  failed  in  the  control  group,  after  a  week  of  no  further  training.  This  may  indicate  that 
a  one-week  period  of  time  is  not  long  enough  to  make  that  determination.  Another, 
analysis  using  descriptive  statistics  was  done  comparing  the  marksmanship  performance 
between  the  baseline  qualification  and  the  retention  qualification.  The  seven-yard  line 
seems  to  indicate  that  the  participants  that  failed  in  the  experimental  group  improved 
more  than  the  participants  that  failed  in  the  control  group.  Refer  the  Figure  10  for  a  bar 
graph  of  the  seven-yard  line  improvement  from  the  baseline  qualification  to  the  retention 
qualification. 
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Figure  10.  Hypothesis  Two  Mean  Difference  in  Seven-yard  Line  Performance. 


The  best  example  of  all  the  participants  that  failed  was  in  the  experimental  group. 
This  participant  failed  the  baseline  qualification  with  a  low  score  of  118  points.  The 
participant  failed  the  baseline  and  the  training  qualification  and  finally  passed  the 
retention  qualification  toward  the  end  of  the  experiment  with  192  points.  It  is  important 
to  remember  that  no  significant  statistical  information  can  be  determined  with  only  one 
person,  but  the  level  of  improvement  is  noteworthy.  Refer  to  Figure  11  for  a  graphical 
representation  of  this  participant’s  performance  over  time.  The  scale  for  the  scores  is 
based  on  points;  more  points  over  time  indicated  better  performance.  The  scale  for  all 
other  marksmanship  performance  data  is  in  centimeters  and  smaller  measurements  over 
time  indicate  better  accuracy. 
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Figure  1 1 .  Most  Improved  Participant 


c.  Hypothesis  Three 

HO:  Participants  in  each  group  will  be  equally  likely  to  maintain  their 
marksmanship  performance  and  scores  on  the  standard  Navy  Handgun  Qualification 
Course  of  fire  (Chief  of  Naval  Operations,  2009)  one  week  after  the  training  day. 

HA:  Participants  in  the  experimental  group  will  be  more  likely  than  those  in  the 
control  group  to  maintain  their  marksmanship  performance  and  scores  on  the  standard 
Navy  Handgun  Qualification  Course  of  fire  (Chief  of  Naval  Operations,  2009)  one  week 
after  the  training  day. 

There  was  no  significant  difference  between  the  control  group  and  experimental 
group  based  on  their  performance  from  the  training  qualification  to  retention 
qualification.  This  experiment  failed  to  reject  the  null  hypothesis  three. 

4.  Exploratory  Analyses 

One  week  with  no  further  training  may  not  be  enough  time  to  determine  any  loses 
of  marksmanship  performance.  According  to  the  qualification  criteria  for  pistols  found  in 
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Enclosure  three,  seetion  five,  paragraph  A  of  the  small  arms  training  and  qualificaiton 
instruetion  (Chief  of  Naval  Operations,  2009)  a  sailor  can  go  as  long  as  30  day  after 
training  before  a  qualification;  “The  period  of  time  between  the 
marksmanship/safety/weapons  familiarization  training  and  live  fire  qualifieation  should 
not  exeeed  30  days.”  (p.  2) 

To  explore  whether  group  differences  in  improvement  oeeurred  between  baseline 
and  the  retention  phase,  two  sample  Mests  assuming  equal  varianees  were  conducted  on 
the  difference  seores  for  eaeh  performance  measure.  Significant  group  differences  in 
improvement  on  the  seven-yard  line  shot  group  {p  =  .02)  and  seven-yard  line  MPI  to  CZP 
ip  =  .003)  were  signifieant.  The  lower  the  diameter  in  the  shot  group,  the  more  consistent 
the  participant  became  over  time.  The  lower  the  length  of  MPI  to  CZP  the  more  aeeurate 
the  partieipant  became  over  time  (refer  to  Table  15).  Thus,  for  both  variables,  negative 
scores  indicate  improvement.  In  both  cases,  the  experimental  group  improved  to  a  greater 
extent  than  the  eontrol  group. 


It-Test:  Two-Sample  Assuming  Equal  Variances  | 

1 

|t-Test:  Two-Sample  Assuming  Equal  Variances  | 

7  Yard  Line  Retention  -  Baseline  Group  Size 

7  Yard  Line  Retention  -  Baseline  MPI  to  CZP 

1  Control  , 

Sim 

contro/ 

Sim 

Mean 

-3.42667 

-11.52 

Mean 

0.16 

-3.11333 

Variance 

93.54495 

125.6746 

Variance 

10.01829 

7.511238 

Observations 

15 

15 

Observations 

15 

15 

Pooled  Variance 

109.6098 

Pooled  Variance 

8.764762 

Hypothesized  Mean  Difference 

0 

Hypothesized  Mean  Difference 

0 

df 

28 

df 

28 

tStat 

2.117061 

tStat 

3.027965 

P(T<=t)  one-tail 

1  0.021635| 

P(T<=t)  one-tail 

1  0.00262l| 

t  Critical  one-tail 

1.701131 

t  Critical  one-tail 

1.701131 

P(T<^)  two-tail 

1  0.043271 1 

P(T<=t)  two-tail 

1  0.005241 1 

t  Critical  two-tail 

2.048407 

t  Critical  two-tail 

2.048407 

SIG  DIFF  between  Control  and  Sim  Group  with  P(T<=t)  two-tail 

SIG  DIFF  between  Control  and  Sim  Group  with  P(T<=t)  two-tail 

SIG  DIFF  between  Control  and  Sim  Group  with  P(T<=t)  two-tail 

SIG  DIFF  between  Control  and  Sim  Group  with  P(T<=t)  two-tail 

Table  15.  Exploratory  Analyses  for  the  Seven  Yard  Line 

The  results  of  this  test  suggest  that  at  least  two  weeks  of  time  is  needed  to  detect 
group  differences  in  improvement. 

B.  SUBJECTIVE  REPORTS  OE  THE  ISMT  EXPERIENCE 

This  section  reviews  input  provided  by  the  participants  of  the  experiment,  after 
the  retention  qualification  was  complete.  Every  participant  was  instructed  to  fill  out  a 
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post  experiment  open-ended  survey.  This  survey  was  an  opportunity  for  the  author  to  get 
a  good  understanding  of  how  the  partieipants  felt  about  the  study  and  marksmanship 
simulators.  Not  every  partieipant  submitted  a  post  survey  and  not  every  partieipant  that 
submitted  a  post  survey  had  mueh  value,  if  any,  toward  eomments  on  their  experience  in 
the  experiment.  The  quotes  that  are  posted  in  this  section  are  their  own  thoughts.  In  an 
attempt  to  protect  the  identity  of  the  participants  in  the  experiment  no  names  shall  be 
used  in  the  section.  Refer  to  Appendix  I  for  a  copy  of  the  post  survey  used  in  this 
experiment. 

1.  Comments  on  the  Value  of  Simulation 

•  Without  any  reservation,  I  fully  believe  in  the  effectiveness  of  the  ISMT  in 
helping  train  and  dramatically  improve  marksmanship  performance...  I 
think  the  SMT’s/EST-2000’s  are  fantastic,  especially  their  ability  to 
simulate  training  scenarios  for  MP’s,  gate  guards,  etc.  I’ve  never  seen  any 
study  proving  beyond  a  doubt  that  these  systems  improve  marksmanship 
“X  percent,”  but  I  have  no  doubt  they’re  worth  the  investment. 

•  In  my  personal  experience,  I  have  had  several  occasions  in  which 
watchstanders  had  to  stand  extra  watches  to  cover  new  shipmates  who 
could  not  get  qualified  on  a  weapon  due  to  being  deployed. 

•  I  whole  heartedly  believe  that  the  ISMT  is  a  useful  tool  for  marksmanship 
training.  The  Sailor  can  get  instant  feedback  with  deficiencies  on  their 
fundamentals  of  marksmanship  and  correct  them  on  the  spot. 

•  I  feel  these  types  of  simulators  are  extremely  useful  in  areas  like 
marksmanship  training.  This  is  a  low  cost  (over  time)  effort  that  can  be 
used  immediately,  unlike  live  fire  ranges  that  have  higher  resourcing 
requirements. 

•  Extremely  useful,  if for  nothing  else  for  the  money  that  would  be  saved  and 
for  the  valuable  information  and  feedback  that  trainees  receive  from  the 
simulator. 

•  ISMT  is  very  useful  for  a  variety  of  marksmanship  training.  First,  it  should 
be  used  to  teach  novice  shooters  before  handling  a  weapon  and  “wasting” 
live  ammunition  to  familiarize  themselves  with  the  “feel”  of  recoil  and 
mechanics  of  a  pistol.  Second,  sustainment  training  should  utilize  the 
ISMT.  Even  if  someone  is  a  good  shot,  skills  deteriorate  with  time.  Using 
the  ISMT  gets  those  skills  back  up  to  standard  before  moving  to  a  live 
range  for  qualification.  Finally,  shooters  of  all  levels  can  benefit  from  the 
feedback  provided  by  the  ISMT.  Expert  performance  can  be  evaluated  and 
emulated  by  others  seeking  to  perfect  their  abilities. 
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•  Use  a  shock  belt  to  keep  the  user  in  a  scared  mindset. .  .for  example  if  the 
weapon  is  condition  1  and  safety  off  but  pointed  in  an  unsafe  direction,  or 
if  the  weapon  is  bolstered  with  the  safety  off,  etc.  Also  it  felt  as  if  the 
pause  between  instruction  and  commencement  of  firing  was  too  long  and 
can  affect  the  results. 

•  Very  useful.  My  score  improved  greatly.  There  is  very  little  marksmanship 
training  in  the  Navy  for  the  typical  sailor  so  it  would  be  a  needed  addition 
to  the  force. 

•  Extremely  useful.  They  would  cut  down  on  costs  dramatically,  allow 
commands  to  tailor  a  program  suited  for  their  members’  needs,  and  allow 
a  safer  environment  to  train  new  shooters  and  re-qualify  experienced 
shooters  while  maintaining  the  fundamentals  of  marksmanship...  Shooters 
will  need  to  feel  some  of  the  aspects  of  shooting  that  cannot  be 
simulatedweapons  failures,  the  shock  of  a  weapon  actually  firing,  and 
some  other  intangible  factors  of  shooting.  However,  simulated  training  can 
drastically  improve  the  efficiency  of  real  training  and  allow  trainers  to 
focus  on  the  areas  that  shooters  are  struggling  on. 

•  Highly  useful.  The  fleet  needs  to  train  on  them  prior  to  wasting  ammo. 
The  usefulness  of  being  able  to  step  back  and  evaluate  your  muzzle 
control  in  invaluable...  not  to  mention  I’m  sure  cost  effective...  This  is  a 
great  concept  that  should  be  used  in  the  fleet,  ft  will  allow  more  bullets  to 
go  to  the  “fight”  and  allow  for  a  more  cost  effective  way  to  train...  and  one 
that  will  be  much  easier  to  get  beneficial  training. 

•  The  indoor  simulator  is  a  must  have,  ft  allows  the  shooter  to  work  on  his 
abilities  and  for  the  instructor  to  have  a  real  world  critique  of  the  errors 
that  the  shooter  is  making. 

•  Yes,  because  it’s  cheaper  and  it  allows  for  real  caption  of  what  a  shooter  is 
doing  wrong. 

•  1  believe  that  this  program  is  needed  for  the  issuing  of  a  firearm  to 
civilians  as  well  as  military  training.  The  better  laws  we  have  and  training 
for  users  of  firearms  the  less  injuries  we  would  see.  There  are  too  many 
Novice  shooters  that  are  more  harm  to  themselves  and  the  innocent  then 
they  are  to  their  targets. 

•  1  think  that  using  the  simulated  trainer  will  be  very  useful  for  training,  ft  is 
a  training  resource  that  can  be  utilized  frequently  simply  due  to  the  fact 
that  it  does  not  require  travel  to  a  range.  The  data  that  is  collected  provides 
useful  feedback  for  the  user  in  terms  of  what  errors  are  being  made  and  at 
what  point  in  the  firing  process  they  need  to  make  corrections. 

•  Simulated  Marksmanship  training  would  be  very  useful  in  providing 
instruction  to  personnel  in  areas/units  that  have  a  lower  need  for  live  fire 
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exercises.  Simulation  can  also  provide  a  safer  and  less  stressful  training 
environment  for  inexperienced  shooters. 

I  believe  that  the  simulators  have  a  place  for  training  and  proficiency.  For 
requals,  they  would  be  good  as  well. 

Comments  of  Real  Versus  Simulated  Training 

Simulators  can  be  used  for  training,  but  at  some  point  the  real  object  must 
be  handled.  Maybe  not  every  qual,  maybe  every  other  or  every  third 
requal.  However,  I  believe  that  with  the  simulators  the  ability  exists  that 
training  is  conducted  more  often,  so  instead  of  yearly  requals,  they  can  be 
conducted  every  6  months,  on  that  time  frame  every  fourth  requal  should 
be  live-fire,  while  the  other  three  can  be  simulated. 

Training  using  this  simulation  could  be  provided  before  the  real  training, 
giving  trainees  the  opportunity  to  absolutely  understand  the  subject  and 
perform  better  and  safer  when  they  are  exposed  to  the  real  training. 

I  do  not  believe  that  simulated  training  will  ever  replace  100  percent  the 
real  training  and  also  to  my  mind  simulation  should  not  replace  the  real 
training.  It  can  only  be  a  supplementary  tool  especially  before  the  real 
training.  To  my  mind,  reality  contains  such  unpredictable  factors  that  no 
simulation  can  predict  and  simulate  them. 

It  is  clear  to  me  that  the  Navy  would  be  afraid  to  replace  real  training  with 
a  sim  and  that  is  unfortunate  because  with  technology  the  way  it  is  now, 
the  ISMT  is  better  than  real  range  training  as  it  provides  immediate 
feedback  based  on  real  data  not  just  opinion. 

Although  I  believe  there  is  a  great  deal  of  value  added  to  using  the 
simulator.  The  real  training  is  still  that.,  ‘real’.  I  believe  that  an  individual 
still  needs  to  experience  the  feel  of  a  real  weapon.  Simulated  training 
however  could  reduce  the  amount  of  real  training. 

If  they  can  be  used  consistently,  I  believe  they  can  be  very  useful.  Seemed 
very  realistic,  minus  wind,  the  smell  of  the  powder,  and  the  bum  of  the 
casings  landing  on  my  arms. 

No,  because  you  can  never  fully  simulate  the  real  thing.  A  shooter  will 
always  need  to  practice  their  shooting  ability  on  the  range  with  live 
rounds.  This  allows  for  a  trae  test  of  the  changes  made  to  improve  the 
shooters  ability  using  the  simulator. 

If  the  exact  conditions  of  “real”  training  were  able  to  be  duplicated  (recoil, 
sound  level,  etc.)  then  I  would  say  that,  yes,  simulated  training  might  be 
used  to  replace  the  range  training.  Never  having  fired  guns  under  the 
“real”  conditions,  I  have  nothing  to  compare  the  simulated  training  to.  I 
feel  confident  however  that  the  simulated  training  would  be  a  great  tool  to 
augment  training  methods  already  in  place. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  question  this  thesis  tried  to  determine  is  if  the  ISMT  ean  be  used  for  the 
purpose  of  improving  participants’  current  marksmanship  performance  and  score  with  the 
goal  of  meeting  a  qualification  standard.  A  second  question  addressed  was  can  ISMT  be 
used  to  sustain  an  acceptable  level  of  performance. 

The  concept  is  simple;  until  a  person  has  received  enough  practice  discharging 
rounds  down  range  to  improve  accuracy  and  proficiency,  it  is  not  certain  that  they  will  be 
effective  with  firearms  during  an  actual  force  protection  crisis.  To  find  the  time  to  take  an 
entire  command  to  a  firing  range  is  difficult  with  the  current  ship  schedule,  unless  it  is  for 
qualification  purposes.  If  the  command  had  a  simulator  onboard  that  could  allow 
personnel  to  practice  marksmanship  skills  on  a  regular  basis,  while  in  port  or  underway, 
then  it  is  reasonable  to  believe  that  personnel  can  and  will  protect  the  ship  during  an 
attack. 

Effective  use  of  a  firearm  is  the  most  important  factor  if  a  threat  was  detected. 
Personnel  are  trained  to  make  the  proper  announcements  to  alert  the  crew  and  give  the 
proper  commands  to  an  attacker,  but  if  the  watch  standers  cannot  hit  the  target  then  they 
have  failed.  A  worse  scenario  is  if  they  miss  the  target  and  hit  a  civilian  or  friendly 
personnel.  By  providing  the  means  to  practice  a  perishable  skill  like  marksmanship,  the 
Navy  may  have  less  incidents  or  casualties.  In  this  chapter,  the  author  describes  logical 
conclusions  based  on  the  results  of  the  data  collected  during  the  experiment. 
Shortcomings  of  the  experiment  are  identified,  and  recommendations  on  how  to  improve 
the  study  and  marksmanship  training  in  general  are  made.  Finally,  future  works  stemming 
from  the  results  are  suggested. 

A.  DISCUSSION 

The  results  indicated  that  simulation  training  improves  certain  aspects  of 
marksmanship  performance,  specifically,  the  seven-yard  line.  Hypothesis  one  addressed 
whether  there  was  a  difference  in  marksmanship  performance  when  current  naval 
marksmanship  training  was  administered  to  the  control  group  and  simulation-based 
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marksmanship  training  was  administered  to  the  experimental  group.  Evidence  for  greater 
improvement  in  the  experimental  group  than  the  control  group  was  found  for  the  seven- 
yard  line  performance  measures.  The  experimental  group  continually  showed  greater 
improvement  on  the  seven-yard  line  shot  group  size  and  the  MPI  to  CZP  length.  The 
seven-yard  line  shot  group  size  and  seven-yard  line  MPI  to  CZP  was  able  to  reject  the 
null  hypothesis  because  the  size  of  the  target  was  much  smaller  than  the  three-yard  line 
target.  The  seven-yard  line  was  a  more  difficult  task  and  recorded  performance  scores 
support  the  assumption  that  there  was  no  ceiling  effect. 

Given  these  results,  it  was  somewhat  surprising  that  significant  group  results  were 
not  found  for  the  15 -yard  line.  The  15 -yard  line  shot  group  size  and  15 -yard  line  MPI  to 
CZP  should  have  been  able  to  test  the  null  hypothesis.  In  this  experiment  it  was  possible 
that  the  participants  were  able  to  get  familiar  with  smaller  targets  from  the  seven-yard 
line  target  exposure.  The  participant  may  have  been  able  to  adjust  their  sight  alignment 
and  sight  picture  well  enough  from  the  three  yard  line  target  to  the  seven  yard  line  target, 
that  it  did  not  make  much  of  a  difference  when  transition  from  a  -seven-yard  line  target 
to  a  15-yard  line  target.  It  was  also  possible  that  the  study  should  be  administered  to  a 
sample  size  of  30-60  participants  to  reduce  the  likelihood  of  type  I  error  from  the  15-yard 
line  hypothesis  testing. 

Hypothesis  two  addressed  whether  there  was  a  difference  in  marksmanship 
performance  among  those  who  do  not  meet  the  qualification  standard  when  current  naval 
marksmanship  training  was  administered  to  the  control  group  and  simulation-based 
marksmanship  training  was  administered  to  the  experimental  group.  Although  there  was 
not  adequate  power  to  test  hypothesis  two  due  to  insufficient  sample  sizes,  the  descriptive 
statistics  suggest  that  simulation  training  may  be  more  beneficial  for  poor  shooters.  The 
margin  for  improvement  once  a  shooter  was  established  as  an  expert  was  very  small. 
Experts  still  showed  improvement  during  the  course  of  the  experiment,  but  the  novice 
shooters  showed  a  substantial  increase  in  marksmanship  performance. 

Hypothesis  three  tested  addressed  whether  there  was  a  difference  in 

marksmanship  performance  after  one  week  passed  from  the  date  the  current  naval 

marksmanship  training  was  administered  to  the  control  group  and  simulation-based 
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marksmanship  training  was  administered  to  the  experimental  group.  The  standards  of  the 
navy  handgun  qualification  may  have  a  lot  to  do  with  the  inability  to  make  a  statistically 
significant  differentiation  in  improvement  between  the  control  group  and  the 
experimental  group.  However,  it  is  important  to  note  that  on  average,  participants  in  both 
groups  showed  improvement  since  the  begirming  to  the  experiment  and  displayed  a  trend 
indicative  of  higher  qualification  scores,  tighter  shot  groups  and  smaller  MPI  to  CZP 
lengths. 

One  possible  reason  for  the  lack  of  significant  results  for  the  three  yard  line  may 
be  due  to  the  simplicity  of  the  distance  and  the  size  of  the  target.  A  majority  of  the 
participants,  regardless  of  training  received,  was  able  to  hit  the  three  yard  target  and  score 
60  of  60  possible  points  for  the  baseline,  training  and  retention  stage  of  the  qualification 
(i.e.,  a  ceiling  effect  for  the  three  yard  line  occurred).  If  a  participant  did  not  gain  all 
possible  points  then  they  were  only  a  few  points  off.  The  shot  group  size  and  length  of 
MPI  to  CZP  would  be  better  determinants  of  improvement;  however  a  ceiling  effect  may 
occur  with  these  performance  measures  as  well.  The  margin  of  improvement  for  all 
measures  on  the  three-yard  line  may  not  be  enough  to  effectively  test  the  null  hypothesis. 

The  next  section  is  recommendations  for  improving  the  experiment  conducted  in 
this  thesis  and  identify  possibilities  of  future  work.  Although  results  are  promising,  there 
were  some  limitations  in  this  study:  small  sample  size,  ceiling  effect,  amount  of  time  to 
test  retention  was  too  small  and  the  marksmanship  ship  training  was  not  as  beneficial  to 
experts.  Future  studies  should  repeat  this  experiment  with  a  much  larger  sample  size.  The 
study  may  benefit  from  a  different  qualification  with  difficult  tasks  and  more  time  should 
be  allotted  when  testing  for  retention  of  marksmanship  skills.  A  majority  of  the  trends 
and  results  of  this  experiment  indicate  a  study  with  only  beginners  and  novice  should 
provide  a  level  of  improvement  to  properly  test  all  hypotheses. 

Research  question  one  did  not  have  any  significant  difference  in  all  categories 
except  for  the  seven-yard  line  shot  group  size  and  the  seven-yard  line  MPI  to  CZP. 
Another  study  should  be  administered  with  a  bigger  sample  size.  This  thesis  only  used  30 
participants  and  divided  them  up  into  two  groups.  A  sample  size  of  only  15  participants 
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in  each  group  may  not  have  been  enough  to  make  a  clear  distinction  of  the  main  effects 
of  the  different  marksmanship  training  stimulus  programs. 

An  analysis  of  the  histograms  for  all  marksmanship  performance  was  conducted 
and  using  non-parametric  results  (Wilcoxin  Rank  Sum  Test)  in  which  the  distributions 
were  not  normal  gave  similar  pattern  of  results.  It  is  reasonable  to  believe  that  if  each 
group  had  30-60  participants  versus  only  15  that  the  histograms  would  have  looked  more 
normal  and  possibly  proved  meaningful  results.  Future  studies  need  much  larger  samples 
in  order  to  correctly  reject  a  false  null  hypothesis.  The  sample  size  in  this  thesis  may  have 
been  too  low  and  result  in  a  high  type  II  error  rate,  incorrectly  retaining  null  hypothesis 
when  it’s  actually  false. 

Because  only  three  participants  failed  the  baseline  qualification,  the  author  was 
unable  to  address  research  question  two.  Another  study  should  be  administered  to 
participants  that  failed  the  baseline  qualification  in  order  to  determine  simulation  training 
is  more  beneficial  to  poor  /  novice  shooters  than  experienced  shooters.  Conducting  the 
same  study  with  beginners  and  failures  at  baseline  will  help  avoid  variability  in  learning 
curves  with  people  that  have  not  used  a  firearm  in  a  while  but  have  experience  with 
firearms  and  formal  marksmanship  training. 

Research  question  three  failed  to  reject  the  null  hypothesis.  The  retention 
qualification  in  this  experiment  was  supposed  to  determine  which  group  would  be  able  to 
retain  their  marksmanship  skills  after  a  week  of  no  training.  Another  analysis  comparing 
the  baseline  qualification  and  the  retention  qualification  may  able  to  test  the  null 
hypothesis  based  on  the  results  of  the  seven  yard  line  shot  group  size  and  seven  yard  line 
MPI  to  CZP.  This  may  indicate  that  another  study  should  be  administered  in  which  the 
retention  phase  is  longer  than  one  week. 

The  navy  regulation  clearly  stated  that  the  time  between  the  training  and  the  live 
fire  qualification  should  not  exceed  30  days.  Thirty  days  may  be  excessive  and  that  a 
study  of  using  navy  pistol  qualification  standards  should  be  conducted  comparing  a  two 
week  and  one  month  time  period  of  no  marksmanship  training.  This  type  of  future  work 
should  be  able  to  determine  if  the  duration  of  time  between  training  and  qualification 
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affect  retention  skill  differentially  based  on  the  type  of  training.  The  next  section  will 
address  all  the  software  technical  difficulties,  system  limitations,  and  pistol  malfunctions 
experienced  during  this  entire  thesis. 

B.  TECHNICAL  DIFFICULTIES  AND  RECOMMENDATIONS 

I.  Failed  to  Load  Magazines  and  Provide  Trace  Profiles 

The  experiment  met  several  challenges.  For  example,  ISMT  was  unable  to  discard 
a  round  when  the  participants  failed  to  shoot  required  rounds  before  '‘cease  fire."'  This 
action  caused  compounding  technical  difficulties:  ISMT  failed  to  load  magazines  with 
the  proper  amount  of  rounds. 

ISMT  failed  to  update  the  number  of  rounds  per  magazine  if  the  participants 
failed  to  fire  all  required  rounds.  The  three-yard  line  and  seven-yard  line  required  two 
magazines  with  six  rounds.  The  15 -yard  line  required  two  magazines  that  had  12  rounds 
in  each  magazine.  If  a  participant  fired  11  of  the  12  rounds  required  for  the  seven-yard 
line,  then  on  the  15-yard  line  the  ISMT  would  reload  with  a  magazine  of  six  rounds  vice 
the  required  12  rounds  per  magazine.  The  ISMT  would  not  reload  with  a  magazine  of  12 
rounds  until  the  participant  fired  one  more  round  at  a  target.  This  is  a  simple  error  that 
deals  with  the  way  ISMT  keeps  count  of  the  number  of  shots  fired. 

An  example  of  this  type  of  error  is  when  a  participant  failed  to  fire  two  rounds  in 
four  seconds  when  the  recorded  instruction  said,  “Cease  fire.”  The  ISMT  was  still 
waiting  for  one  more  round  to  fulfill  the  requirement  of  12  rounds  for  the  three  and  seven 
yard  line.  The  system  was  not  programed  to  count  that  missed  round  as  a  zero  and 
continue  to  calculate  the  marksmanship  performance  data  for  the  11  of  12  rounds  that 
were  actually  fired.  ISMT  also  does  not  provide  the  simulation  user  a  button  to  push  or 
have  some  way  to  manually  instruct  the  ISMT  to  continue  the  qualification  and  count  the 
missing  round  as  a  zero  for  points.  Surprisingly,  when  the  participant  fired  all  12  rounds 
at  the  three  or  seven  yard  line  target  but  at  the  15 -yard  line  only  hit  the  target  23  of  24 
times  then  ISMT  would  provide  the  needed  marksmanship  performance  data  at  the  end  of 
the  qualification. 
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2. 


Provide  Trace  Profiles 


The  experiment  met  several  challenges  first,  ISMT  was  unable  to  discard  a  round 
when  the  participants  failed  to  shot  required  rounds  before  cease  fire.  This  action  caused 
an  inability  to  provide  trace  profiles  on  several  occasions  and  the  rapid-fire  sequence  was 
suspected  to  be  the  cause.  The  navy  handgun  qualification  had  a  series  of  instructions  that 
required  the  participants  to  shoot  rapid  fire  from  the  holster.  For  example,  the  recorded 
instruction  may  tell  the  participant  to  shoot  two  rounds  in  four  seconds.  It  was  determined 
that  ISMT  was  unable  to  track  the  barrel  of  the  pistol  effectively.  The  M9  Berretta  was 
equipped  with  an  infrared  beam  at  the  end  of  the  barrel  and  the  ISMT  had  a  hit  camera  to 
trace  where  the  pistol  was  pointed  towards  on  the  projector  screen.  Once  the  recorded 
instruction  says  fire,  the  participants  quickly  drew  the  weapon  and  fire  two  rounds.  After 
the  qualification  was  complete,  the  instructors  collected  the  marksmanship  performance 
data  and  noticed  that  the  trace  profile  feature  did  not  always  work. 

The  ISMT  had  four  different  trace  profile  failures.  First  the  trace  profile  would 
simply  display  no  trace  data.  The  second  failure  was  when  the  trace  profile  did  not 
display  the  green  one  second  before  shot  and  the  purple  .2  seconds  before  shot,  but  would 
display  the  blue  one  second  after  shot.  The  third  failure  was  when  the  trace  profile  did  not 
display  the  green  one  second  before  shot,  but  did  display  the  purple  .2  seconds  before 
shot  and  the  blue  one  second  after  shot.  The  last  failure  was  when  the  trace  profile  did 
display  the  green  one  second  before  shot,  but  not  the  purple  .2  seconds  before  shot  and 
the  blue  one  second  after  shot. 

The  author  kept  track  of  every  time  the  trace  profile  had  a  malfunction.  He 
recorded  every  observation  in  Excel  and  at  the  end  of  the  experiment  had  4,224 
observations.  A  quick  calculation  indicated  that  the  trace  profile  failed  to  work  properly 
17  percent  of  the  time.  The  ISMT  tracking  system  should  be  modified  so  that  it  can  track 
the  barrel  sensor  faster  in  order  to  help  the  instructor  to  train  shooters  while  using  the 
trace  profile  feature. 
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3.  Qualification  Scenarios 

The  creations  of  the  qualification  scenarios  took  several  trials.  The  first  trial  had  a 
separate  target  for  each  course  of  fire.  For  example,  the  first  set  of  instructions  on  the 
three-yard  line  had  the  participant  fire  two  rounds  in  four  seconds.  A  target  would  appear 
before  the  recorded  instructions  said  “Fire.”  Then  the  target  disappeared  when  the 
recoded  instructions  said  “Cease  fire.”  A  new  target  would  appear  for  the  next  set  of 
instructions.  This  scenario  failed  to  provide  any  shot  group  size  data  at  the  end  of  the 
qualification  because  ISMT  could  not  calculate  a  shot  group  with  only  two  rounds  for  a 
target.  This  issue  was  fixed  by  creating  a  scenario  that  had  only  three  targets  representing 
the  three,  seven  and  15-yard  line.  The  targets  would  appear  and  disappear  at  the 
beginning  and  end  of  the  assigned  stage  of  the  qualification.  This  method  enabled 
instructors  to  record  all  marksmanship  performance  data  that  ISMT  can  provide. 

4.  Loss  of  Marksmanship  Performance  Data 

Another  issue  was  that  the  scenario  would  not  provide  marksmanship 
performance  data  if  a  participant  failed  to  fire  all  required  rounds  for  each  stage  of  the 
qualification.  The  navy  handgun  COF  required  the  participants  to  fire  12  rounds  at  the 
three-yard  line  target,  12  rounds  at  the  seven  yard  line  target  and  24  rounds  at  the  15-yard 
target.  If  a  participant  only  fired  11  of  the  12  rounds  required  for  the  three-yard  line 
target,  then  ISMT  output  would  state  invalid  for  shot  group  size  and  MPI  to  CZP.  The 
instructor  would  observe  invalid  for  performance  measures  if  the  same  action  occurred  on 
the  seven-yard  line.  Surprisingly,  ISMT  would  provide  marksmanship  data  if  the  same 
action  occurred  on  the  1 5 -yard  line. 

The  author  figured  out  a  way  to  improve  the  scenario  and  take  the  limitations  of 
ISMT  under  consideration.  A  future  study  should  create  a  scenario  exactly  like  the 
scenario  created  in  the  experiment  except  with  a  backup  target  on  the  side  that  is  smaller 
and  different  looking  from  the  main  target.  When  the  participant  failed  to  fire  required 
rounds  within  the  standard  time  limit,  then  the  instructor  should  instruct  the  participant  to 
fire  a  round  at  the  smaller  and  different  looking  target  located  off  to  the  side.  The 
instructor  should  keep  track  of  which  round  was  fired  so  that  upon  completion  of  the 


61 


qualification  the  instructor  can  drop  that  particular  shot.  ISMT  would  recalculate  all  the 
measurements  for  the  shot  groups  and  MPI  to  CZP  data.  However,  ISMT  will  not 
recalculate  the  score  and  special  measures  should  be  taken  to  prevent  participants  from 
firing  at  the  target  after  the  command  to  cease  fire. 

5.  Inability  to  Calculate  Qualification  Scores 

It  also  was  vital  that  the  participant  fired  the  round  away  from  the  actual  target 
used  for  the  qualification  because  ISMT  would  not  update  the  score  when  the  instructor 
dropped  the  shot  upon  completion  of  the  qualification.  For  example,  a  participant  only 
fired  one  round  versus  two  in  four  seconds  and  the  author  told  the  participant  to  fire  the 
round  at  the  main  target  instead  of  the  backup  target.  Then  at  the  end  of  the  qualification 
when  the  author  drops  that  particular  shot,  ISMT  will  not  subtract  the  points  earned.  This 
limitation  will  falsify  the  actual  score  that  the  participant  should  have  earned  in  real  life 
because  the  simulation  was  not  smart  enough  to  recalculate.  It  is  important  to  have  the 
actual  and  the  backup  targets  look  very  different  and  located  as  far  apart  from  each  other 
as  possible  to  ensure  that  the  participants  do  not  get  confused  with  the  instructions. 

The  author  has  not  tried  a  second  possible  solution.  Instead  of  having  two  separate 
targets,  if  the  participant  were  instructed  to  fire  a  round  pointed  away  from  the  projector 
screen,  then  ISMT  may  be  able  to  continue  with  the  qualification.  The  instructor  should 
be  able  to  drop  the  shot  at  the  end  of  the  qualification  and  have  no  problems  with  not 
being  able  to  recalculate  the  score.  Future  experiments  should  try  to  implement  these 
solutions  in  a  pilot  study  before  conducting  their  actual  experiment. 

To  resolve  this  limitation  in  the  experiment  for  this  thesis,  the  author  had  to  rely 
on  previous  marksmanship  qualification  experiences  and  consider  this  error  as  a 
technicality.  The  author  did  not  derive  these  solutions  until  after  the  experiment  had 
already  began  and  did  not  have  time  to  restart  another  experiment.  The  instructor 
considered  this  particular  technicality  as  an  alibi,  which  meant  the  participant  was 
allowed  to  fire  the  round  in  a  reasonable  allotted  time. 
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6.  Error  Messages 

The  author  lost  performance  data  from  error  messages  while  trying  to  print  the 
average  shot  group  results  page.  The  author  also  lost  performance  data  from  error 
messages  of  which  the  cause  was  unknown.  The  error  messages  during  the  experiment 
only  came  at  the  end  of  a  qualification  and  during  data  collection.  The  author  was  not 
sure  what  caused  the  first  error  message  but  took  a  picture  for  documentation  purposes.  It 
is  important  to  note  that  when  the  instructor  pressed  the  close  button  that  all  data  was  lost 
and  irretrievable.  Refer  to  Figure  12  to  view  the  error  message. 


Figure  12.  MMI  Lane  Resource  Applications  Error 

ISMT  also  had  printing  issues  that  caused  an  error  message,  which  resulted  in  loss 
of  data.  At  the  end  of  every  qualification,  the  instructors  would  have  to  print  out  the 
marksmanship  performance  data  as  backup  in  case  electronic  soft  copies  were  corrupted 
or  lost.  This  particular  ISMT  failed  to  print  the  actual  size  of  the  average  shot  group  size. 
ISMT  at  first  always  printed  the  average  shot  group  size  as  1 .0  centimeters.  The  author 
manually  wrote  the  actual  size  on  the  print  out  and  filed  it.  This  limitation  continued  until 
it  was  discovered  that  the  ISMT  would  display  an  error  message  when  the  instructor  tried 
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to  print  the  average  shot  group  results.  The  error  message  only  gave  one  button  to  push 
and  ISMT  would  exit  the  simulation  screen  and  all  data  was  lost.  Refer  to  Figure  13  to 
view  the  LANEDX  MFC  Application  error  message. 


MFC  Application 


MFC  Application  has  encounteted  a  problem  and 
needs  to  close.  We  aiej  sorry  for  the  inconvenience. 


If  you  vvere  in  Ihe  middle  of  something,  the  information  you  were  working  on 
might  be  lost. 


For  more  information  about  this  error,  click  here 


Figure  13.  LANEDX  MFC  Application  Error 


The  instructor  decided  to  press  click  here  versus  the  close  button  and  another 
message  came  up.  The  author  is  not  sure  what  the  message  means  but  took  a  picture  for 
documentation  purposes.  Refer  to  Figure  14  to  view  the  error  signature  message. 


Figure  14.  Error  Signature  Message 

The  instructor  once  again  decided  to  press  click  here  versus  the  close  button  and 

another  message  came  up.  The  author  is  not  sure  what  the  message  means  but  took  a 
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picture  for  documentation  purposes.  Refer  to  Figure  15  to  view  the  error  signature 
message. 


Figure  15.  Error  Report  Contents 


The  author  found  no  solutions  for  this  error  but  came  up  with  a  policy  that  future 
thesis  studies  should  consider.  After  completion  of  any  qualification,  the  instruetors 
should  first  record  all  data  onto  an  Excel  file  and  be  extremely  careful  not  to  print 
anything  out.  The  instructors  should  then  take  a  picture  of  every  data  screen  which 
includes  the  following;  results  page,  average  shot  group  size,  shot  group  one  (three-yard 
line),  shot  group  two  (seven  yard  line),  shot  group  three  (15-yard  line),  MPl  to  CZP 
group  one  (three-yard  line),  MPl  to  CZP  group  two  (seven-yard  line),  MPl  to  CZP  group 
three  (15-yard  line),  and  all  48  trace  profiles.  The  author  also  required  that  all  data 
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screens  get  printed  with  the  exception  of  the  average  shot  group  page  because  it  was  the 
source  of  the  error  messages. 

7.  Menu  Navigation 

The  next  technical  difficulty  was  caused  by  the  instructor  that  ran  the  simulation. 
In  order  to  navigate  through  the  ISMT  interface,  the  user  had  to  either  push  enter  or  the 
letter  Q  in  order  to  back  out  of  a  screen.  The  technical  difficulty  happens  at  the  end  of  the 
qualification  when  the  instructors  are  collecting  data.  If  the  user  pushes  the  enter  button 
and  the  simulation  has  '^Continue"  selected,  then  the  ISMT  will  move  to  the  next  screen 
and  discard  all  the  marksmanship  performance  data.  It  is  critical  that  all  data  is  recorded 
in  Excel,  all  pictures  are  taken  and  all  data  screens  are  printed  out  before  the  Continue 
option  is  selected  and  the  enter  button  is  pushed.  There  is  no  way  to  undo,  or  go  back  to 
previous  screen  to  enable  the  instructors  to  collect  data  again.  The  same  effect  will  occur 
if  the  simulation  user  pushes  the  Q  button  at  any  time.  To  avoid  any  loss  of  data,  every 
instructor  must  be  conscious  of  this  limitation  and  aware  of  which  option  they  are  about 
to  select  on  the  screen. 

8.  Malfunction  of  the  M9  Berettas 

The  last  technical  difficulty  and  the  biggest  issue  for  this  thesis  were  the  seven  out 
of  commission  (OOC)  M9  Berettas.  The  experiment  started  out  with  two  M9  Berettas  and 
only  used  one  to  conduct  the  experiment.  The  other  was  used  as  a  prop  to  help  the 
instructor  demonstrate  the  functionality  of  the  pistol  to  the  participants.  Given  the  number 
of  participants  and  the  length  of  each  session,  this  was  a  lot  of  wear  and  tear  on  the 
pistols.  Training  sessions  began  at  0700  in  the  morning  and  the  last  training  session 
ended  at  1 800  in  the  evening. 

The  first  pistol  with  serial  number  BER4484182  had  an  automatic  slide-lock 

activating  mechanism  break.  This  caused  the  pistol  to  not  lock  the  slide  to  the  rear  when 

the  magazine  was  empty.  The  second  pistol  with  serial  number  BER441064Z  had  a 

safety-on  sensor  that  was  too  slow.  This  caused  the  pistol  to  randomly  and  inadvertently 

fire  if  the  hammer  is  in  the  single  action  position  (in  the  full  cocked  position)  and  the 

safety  was  engaged  to  ride  the  hammer  home.  The  ISMT  still  thinks  that  the  safety  level 
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was  off  because  the  safety-on  sensor  was  too  slow  and  the  hammer  reached  the  forward 
position  before  ISMT  had  a  chance  to  catch  up  causing  the  pistol  will  fire  a  round.  The 
pistol  with  serial  number  BER441064Z  also  fired  a  round  when  the  participant  was 
chambering  a  virtual  round  by  pulling  on  the  slide  to  the  rear  and  releasing  the  slide 
forward. 

The  third  pistol  with  serial  number  BER408276Z  did  not  detect  that  the  hammer 
had  fallen  and  would  not  fire  a  round.  The  forth  pistol  with  serial  number  BER422910Z 
had  a  slow  safety-on  sensor  and  exhibited  the  same  symptoms  as  the  pistol  with  serial 
number  BER441064Z.  The  fifth  pistol  with  serial  number  BER426414Z  often 
mysteriously  lost  virtual  rounds  from  the  magazine  without  firing  the  pistol  or  without 
ejecting  a  round  from  the  chamber.  The  simulation  user  watched  the  screen  and  witnessed 
the  rounds  in  the  magazine  decrease  while  the  weapon  was  not  being  fired. 

The  sixth  pistol  with  serial  number  BER412636Z  had  an  automatic  slide-lock 
mechanism  that  would  only  partially  engage.  This  would  cause  the  pistol  to  release  the 
slide  forward  with  the  slightest  tap  or  rapidly  moving  the  pistol  in  a  forward  motion  and 
stopping.  The  pistol  was  an  ineffective  training  tool  because  the  participants  never 
experienced  the  motion  of  have  to  press  the  slide  release  lever  to  close  the  slide  since  the 
pistol  did  that  for  them.  The  last  pistol  with  serial  number  BER426445Z  had  a  slow 
safety  sensor-on  sensor,  which  had  the  same  symptoms  as  the  previous  pistols  with  the 
same  malfunction  and  would  not  fire  all  rounds  in  the  magazine. 

In  conclusion,  based  on  the  vast  number  of  technical  difficulties,  the  main 
recommendation  for  future  work  is  to  have  several  data  backup  systems  in  place  and  have 
as  many  pistols  available  as  possible.  Additional  recommendations  for  future  work 
should  consist  of  studies  that  only  focus  on  novice  shooters  and  have  much  larger  sample 
sizes.  A  study  with  a  more  challenging  qualification  than  the  one  used  in  the  experiment 
should  be  considered  in  order  to  avoid  any  ceiling  affects. 

C.  SUMMARY 

This  thesis  found  significant  differences  between  the  current  naval  marksmanship 
training  program  and  a  simulation-based  marksmanship  training  program.  The  results 
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from  the  seven-yard  line  portion  of  the  navy  handgun  qualification  COF  indicated  that 
participants  significantly  benefited  from  the  simulation-based  training  program  more  so 
than  the  current  naval  marksmanship  training  program.  An  exploratory  analysis  suggests 
that  a  time  period  of  at  least  two  weeks  is  necessary  to  detect  group  differences  in 
retention  of  marksmanship  skills.  Additionally,  an  exploratory  analysis  indicates  that  the 
simulation-based  training  program  may  be  most  beneficial  for  poorer  marksmen.  Finally, 
responses  from  the  post  experiment  survey  indicated  that  the  majority  of  participants  in 
the  experiment  still  had  reservation  about  replacing  real  live  fire  training  with  simulation- 
based  training,  but  established  that  there  are  benefits  to  having  a  marksmanship  trainer 
available. 
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APPENDIX  A.  OPNAVINST3591.1F  STAGE 


The  contents  of  this  appendix  are  screen  captures  of  the  OPNAVrNST3591.1F 
stage.  These  pictures  are  provided  so  that  future  thesis  students  can  navigate  to  the  stage 
used  in  this  experiment.  Readers  may  also  reference  the  fats  manual.  The  stage  was 
named  after  the  OPNAV  INSTRUCTION  3591. IF  Small  Arms  Training  and 
Qualification. 


Lanes  Training 
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Lanes  Training 


OK 


Cancel 


1  Lane  Courses 

2  Lane  Courses 

4  Lane  Courses 

5  Lane  Courses 


Lanes  Authoring 


Lanes  Authorizing 


Open  Courses  Tab 
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Open  Courses 


Errors  and  Warnings 


Ready 


OPNAVINST3591.1F  Stage 
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THIS  PAGE  INTENTIONALLY  LEFT  BLANK 
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APPENDIX  B.  3-YARD  ZERO  SCENARIO 


The  contents  of  the  appendix  are  screen  captures  of  the  3-yard  zero  scenario. 
These  pictures  are  provided  so  that  anyone  can  recreate  the  course  of  fire  used  in  this 
experiment.  The  scenario  was  established  with  respect  to  Navy  Handgun  Qualification 
Course  found  in  the  OPNAV  INSTRUCTION  3591. IF  Small  Arms  Training  and 
Qualification. 


Sub  Stage 
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Course  Name 


Course  Standards 
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Practice  Type 


Practiee  Weapon 
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^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


1-Lane  USER  M9  Courseware  [ 


:  Project  ^it  View  Tools  Window  Help 

i  ✓  y  '-00%  ~^\JL  ^ 


9  OPNAVINST3591.1F 
r  ^  Verl  Navy  Handgun  Qual  Co 
S  ^  VerlS  Navy  Handgun  Qual  C 
S  ^  Unlimited  Practice 
S  ^  Ver2  Navy  Handgun  Qual  Co 
r  ^  Ver2S  Navy  Handgun  Qual  C 
-  ^  Zero  3  Yard  Line 

S  ^  Boresight  3  Yard  Line 
ti  0  M9ZER03 
-  ^  Subserial 

0  FATSZERO 
S  ^  Zero  7  Yard  Line 
r  ^  Zero  15  Yard  Line 


I  Type  II  Weapon  |  Ammunition*  |  PreparatW^ ||  Runtime*  ||  Group  Assignment*  ||  Shot  Standards  ||  Standards  ||  Weapon  Sensor  Restrictions  ||  Scoring  ||  Results*  | 


This  tab  defines  this  practice's  ammunition  quantity  and  characteristics. 

Ammunition  load: 


1  Add  1  h  ol 

1  Update  I  |5  Z\ 

\  Remove  | 

1  Move  Up  1 

[Move  Downj 

^  magazine(s)/bell(s)  containing 
^  round(s)  per  magazine/belt. 


3  T  otal  rounds  added  by  this  practice. 

r~]  Override  the  default  weapon  ammunition  for  this  practice. 
Ammunition  type  n  Ball 


Define  Ammunition  on  a  Per  Round  Basis 
0  Use  tracer  effects. 


I _ Define  Tracers  on  a  Per  Round  Basis _ j 

0  Use  any  rounds  left  over  from  the  previous  practice  in  this  course  b( 
ammunition  added  for  this  practice. 

r~l  Limit  the  number  of  shots 

Number  of  allowed  shots  |l  zl 
0  Use  sighters  at  the  beginning  of  the  practice. 

Number  of  sighters  [l^ _ 


_< _ > 

[  Errors  and  Warnings  | 

Ready  NUM  SCRL 

Practice  Ammunition 


Practice  Preparation 
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Practice  Runtime 


Practice  Group  Assignment 
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[  Errors  and  Warnings  ] 

Ready  NUM  SCRL 


Practice  Shot  Standards 


Practice  Standards 
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[  Errors  and  Warnings 

Ready 


Practice  Weapon  Sensor  Restrictions 


^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

BOfxil 

:  Project  Wit  View  Tools  Window  Help 

/  1-Lane  USER  M9  Courseware  | 

<1  >  X 

Q  0  OPNAVINST3591.1F 

ai  (§)  Verl  Navy  Handgun  Qual  Co 
S  (§)  VerlS  Navy  Handgun  Qual  C 
S  Unlimited  Practice 

S  (§)  Ver2  Navy  Handgun  Qual  Co 
S  (§)  Ver2S  Navy  Handgun  Qual  C 
B  Zero  3  Yard  Line 

B  ^  Boresight  3  Yard  Line 
B  0  M9ZER03 
B  O  Subserial 

0  FATSZERO 
a  0  Zero  7  Yard  Line 
a  0  Zero  15  Yard  Line 


< _ > 

I  Errors  and  Warnings  | 

Ready  NUM  SCRL 


I  Type  II  Weapon  ||  Ammunition*  ||  Preparation*  ||  Runtime*  1|  Group  Assignment*  ||  Shot  Standards  ||  Standards  ||  Weapon  Sensor  Restrictions  |  Scoring  |  Results*  | 

This  tab  defines  how  a  practice  is  scored. 

Scoring  method 


By  hits  in  target  region[s) 


Highest  possible  score 

r° " 

When  the  practice  score  is  combined  into  the  course  score,  it  is  multiplied  by  a  factor  of 

ElZJ 

When  using  a  group  size  to  determine  whether  the  practice  standard  is  met 
Use  the  maximum  fired  group  size.  v 


Practice  Scoring 
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Practice  Results  Top 


Practice  Results  Bottom 
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Background  category 
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Sub  Serial  Scene 
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Ready  NUM  SCRL 

Sub  Serial  Ballistic  Terrain  Lines 
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S  0  Subserial 

0'  FATSZERO 
0  Zero  7  Yard  Line 
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I  Errors  and  Warnings  | 

Ready 


[  Preparation^!  Scene*  ||  Ballistic  Terrain  Lines*  |  Runtime*  |  previei^ 


This  tab  is  used  to  set  options  used  during  the  running  of  this  subserial. 

Subserial  runs  until 


I  practice  is  fired  or  all  targets  have  timed  out 


shots 


[~~l  Delay  the  display  of  shot  results  in  the  immediate  feedback  area. 
\i  ]  econds 

□  Delay  subsetial  end  by 
I  ~|  •  conds 

Wind  flag  is  displayed 


I  Only  if  the  wind  speed  is  not  zero 


Wind  flag  is  drawn  with  a 
T  ransparent  background 
Wind  flag  is  located  in  the 


Lower  left  corner 
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Sub  Serial  Runtime 
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I  Errors  and  Warnings 

Ready 


Sub  Serial  Preview 


Target  Details 
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:  Project  Wit  View  Tools  Window  Help 


lag  V*  B  Zoom  100%  ^  ^  -’i  I  4t5  !  0  (§)  ^  ^  B 
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t  ^  Verl  Navy  Handgun  Qual  Co 
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a  0  M9ZER03 
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^  ^  Zero  7  Yard  Line 
I-  ^  Zero  15  Yard  Line 


I  Details*  fj  Lifetime*  1 1  Position*  ||  Behaviors*  ||  Scoring*  ||  Regions*  | 


This  lab  configures  target  exposure  timing. 

0  The  target  appears 

[3.0 _ J  seconds  after  the  subserial  begins. 

O  The  target  appears  after  a  random  amount  of  time. 

M  inimum  time  M  aximum  time 

1 3.0  ]  seconds  after  the  subserial  begins  |3.U  ]  seconds  after  the  subserial  begins 

0  The  target  disappears 

[900.0  ~~[  seconds  after  it  appears. 

O  The  target  disappears  after  a  random  amount  of  time. 

M  inimum  time  M  aximum  time 

[30.0  ~|  seconds  after  it  appears  |60.0  ~|  seconds  after  it  appears 

O  The  target  disappears  when  its  path  ends. 

O  The  target  disappears  when  the  subserial  ends. 


Errors  and  Warnings  | 

Ready 


Target  Lifetime 


B  0  OPNAVINST3591.1F 

a)  0  Verl  Navy  Handgun  Qual  Co 
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S  0  Ver2  Navy  Handgun  Qual  Co 
Q  0  Ver2S  Navy  Handgun  Qual  C 
B  0  Zero  3  Yard  Line 

e  Boresight  3  Yard  Line 
B  0  M9ZER03 
H  0  Subserial 

0  FATSZERO 
+  0  Zero  7  Yard  Line 
B  0  Zero  15  Yard  Line 


I  Details*  ||  Lifetime*  |  Position*  |  Behaviors*  ||  Scoring*  ||  Regions*  | 

O  The  target  travels  around  the  path  specified  below  a  limited  number  of  times  before  stopping. 

Q _ ^  loops 

0[The  target  tiaveh  atound  the  path  specified  below  an  infinite  number  of  times.] 

[  Add  I 

I  Update  ~) 

[  Move  Up~| 

[Move  Down] 

2^  ^  X  Position  0Y  Position  Range  Travel  time  to  next  node  [  Remove  j 

<1-  O  -|>  [?10  ]  [525  I  [274  |m  [0.0  ~|*  0  Visible 

|s^  ^  The  target  is  stationary. 


X  Y  Z  Time  Visible 

510  525  2.74  0.00  Visible 


Target  Position 
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Target  Scoring 
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BPS 


I^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


i  Project  ^it  View  Tools  Window  Help 


i  eg  V"  y  Zoom  100%  ^  I JL %i  ' _ 0  ©  ^ 

1-Lane  USER  M9  Courseware  ) 


Target  Regions 
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APPENDIX  C.  7-YARD  ZERO  SCENARIO 


The  contents  of  the  appendix  are  screen  captures  of  the  7-yard  zero  scenario. 
These  pictures  are  provided  so  that  anyone  can  recreate  the  course  of  fire  used  in  this 
experiment.  The  scenario  was  established  with  respect  to  Navy  Handgun  Qualification 
Course  found  in  the  OPNAV  INSTRUCTION  3591. IF  Small  Arms  Training  and 
Qualification. 
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[  Errors  and  Warnings  ] 

Ready  NUM  SCRL 


Course  Name 


^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

i  Project  Wit  View  Tools  Window  Help 

i  V"  y  :ij  ^ ;  fs  4is  1 9  0  1 

/  1-Lane  USER  M9  Courseware  | 

<1  >  X 

i  0  OPNAVINST3591.1F 

+  0  Verl  Navy  Handgun  Qual  Co 
+  0  VerlS  Navy  Handgun  Qual  C 
+  0  Unlimited  Practice 
+  0  Ver2  Navy  Handgun  Qual  Co 
H-  0  Ver2S  Navy  Handgun  Qual  C 
+  0  Zero  3  Yard  Line 
-  0  Zero  7  Yard  Line 

-  0  Boresight  7  Yard  Line 
r  0  M9ZERO 
-  0  Subserial 

0  FATSZERO 
|£  (§)  Zero  15  Yard  Line 


I  Course  Name*  |  Course  Standards*  | 


This  tab  defines  criteria  required  to  pass  this  course. 

Piimaiy  course  standard  to  use 


Secondary  course  standard  t( 

[Ot £1  I  None 


I  Errors  and  Warnings  | 

Ready 


Course  Standards 
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Practice  Type 


^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

:  Project  Mit  View  Tools  Window  Help 

irsvH  1 

1 -Lane  USER  M9  Courseware  ] 

<1  >  X 

OPNAVINST  3591 .  IF  |  Type  ]  Weapon  |  Ammunition*  1|  Preparation*  ||  Runtime*  f  Group  Assignment*  ||  Shot  Standards  1|  Standards  f  Weapon  Sensor  Restrictions  ||  Scoring  1[  Results*  | 

Verl  Navy  Handgun  Qual  Co  |  - 


i 

B# 


\  VerlS  Navy  Handgun  Qual  C 
_J  Unlimited  Practice 
S  0  Ver2  Navy  Handgun  Qual  Co 
H  #  Ver2S  Navy  Handgun  Qual  C 
I-  ^  Zero  3  Yard  Line 
-  ^  Zero  7  Yard  Line 

-  ^  Boresight  7  Yard  Line 
a  0  M9ZERO 
a  ^  Subserial 

0  FATSZERO 
ti  0  Zero  15  Yard  Line 


Errors  and  Warnings  ] 

Ready 


This  tab  overrides  certain  physical  options  relating  to  the  weapon  and  its  orientation  to  the  shooting 
lane. 


HUiOverride  the  second  zero-zero  range 
|l  Meters 

□  Override  the  weapon  sight 


□  Override  the  distance  to  the  screen  /  screen  width  ratio.  For  correct  target  si 
2  (a  6  meter  firing  line  is  correct  for  a  3  meter  wide  screen). 


□  This  weapon  uses  a  second  sight  for  this  practice.  The  second  sight  offset  is: 

Horizontal  offset 

|0  c]  Millimeters 

Vertical  offset 
|0  C]  Millimeters 

□  Use  the  sub  weapon  laser  to  generate  trace  information.  (Weapon  must  support  sub  weapon  trace 
to  use  this  option.) 


Practice  Weapon 
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[  Errors  and  Warnings  ] 

Ready  NUM  SCRL 


Practice  Ammunition 


Practice  Preparation 
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BBBI 


^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


Project  yit  View  Tools  Window  Help 

i  ✓  y  '-00%  V||  X.  ^  " 


y^ 1-Lane  USER  M9  Courseware  [ 


>  -^y 


9  OPNAVINST3591.1F 
r  ^  Verl  Navy  Handgun  Qual  Co 
S  ^  VerlS  Navy  Handgun  Qual  C 
S  ^  Unlimited  Practice 
r  ^  Ver2  Navy  Handgun  Qual  Co 
I-  ^  Ver2S  Navy  Handgun  Qual  C 
r  ^  Zero  3  Yard  Line 
-  ^  Zero  7  Yard  Line 

S  ^  Boresight  7  Yard  Line 
1=1  0  M9ZERO 
B  ^  Subserial 

0  FATSZERO 
S  0  Zero  15  Yard  Line 


Type  II  Weapon  ||  Ammunition*  ||  Preparation*  |  Runtime*  |  Group  Assignment*  ||  Shot  Standards  ||  Standards  ||  Weapon  Sensor  Restrictions  ||  Scoring  ||  Results*  | 


This  tab  sets  options  that  are  used  during  the  running  of  the  practice. 

OiShow  immediate  feedback  at  the  top  of  the  lane 

Show  the  MPI  of  each  burst  as  a  cross  in  the  immediate  feedback  area. 
r~1  Show  bullet  holes  in  targets. 

r~1  Show  target  muzzle  flashes  (one  per  second,  all  targets). 

PI  Show  splashes  (ricochets)  when  rounds  impact  off  target. 

1 100  y]  X  chance  of  splash  on  any  given  shot  (use  1 0OX  to  show  for  each  round) 

n  Set  the  wind  direction/speed  and  weather/ view. 

Wind  originates  from 
IZTOocK 
At  a  speed  of 
|0  -tlKph 
Wind  speed  and  direction  varies  by 


The  weather/view  is 
[  Normal 

With  a  visibility  of 
[2000  I  Meters 

0  Delay  practice  end  by 
[2.0  I  seconds 

r~1  Show  target  identifiers 


[  Errors  and  Warnings  j 

Ready 


Practice  Runtime 


Practice  Group  Assignment 
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Practice  Shot  Standards 


Practice  Standards 
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[  Errors  and  Warnings  ] 

Ready  NUM  SCRL 


Practice  Weapon  Sensor  Restrietions 


Cranes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

:  Project  ^it  View  Tools  Window  Help 

is^  ✓  H  ■  "  a 

1-Lane  USER  M9  Courseware  | 

<  >  X 

Q  0  OPNAVINST3591.1F 

Verl  Navy  Handgun  Qual  Co 
^  VerlS  Navy  Handgun  Qual  C 
^  Unlimited  Practice 
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i  Ver2S  Navy  Handgun  Qual  C 
^  Zero  3  Yard  Line 
(  Zero  7  Yard  Line 

Boresight  7  Yard  Line 
a  0  M9ZERO 
Q  0  Subserial 

0  FATSZERO 
hi  0  Zero  15  Yard  Line 


S  0  2 

30  Z 


I  Type  II  Weapon  ||  Ammunition*  ||  Preparation*  ||  Runtime*  ||  Group  Assignment*  ||  Shot  Standards  ||  Standards  ||  Weapon  Sensor  Restrictions  |  Scoring  |  Results*  | 

This  tab  defines  how  a  practice  is  scored. 

Scoring  method 

Highest  possible  score 

When  the  practice  score  is  combined  into  the  course  score,  it  is  multiplied  by  a  factor  of 

1^1 

When  using  a  group  size  to  determine  whether  the  practice  standard  is  met 


I  Errors  and  Warnings  | 

Ready 


I  Use  the  maximum  fired  group  size. 


Practice  Scoring 
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Practice  Results  Top 


Practice  Results  Bottom 
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I  Errors  and  Warnings 

Ready 


Sub  Serial  Preparation 


Sub  Serial  Scene 
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Errors  and  Warnings  j 

Ready  NUM  SCRL 


Sub  Serial  Ballistic  Terrain  Lines 
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Q  0  M9ZERO 
Q  ^  Subserial 

0  FATSZERO 
(§)  Zero  15  Yard  Line 


Errors  and  Warnings  | 

Ready 


[  Preparation*  |  Scene*  ||  Ballistic  Terrain  Lines*  |  Runtime*  |  Previei^ 


This  tab  is  used  to  set  options  used  during  the  running  of  this  subserial. 

Subserial  runs  until 


^  shots 


[~~l  Delay  the  display  of  shot  results  in  the  immediate  feedback  area, 
conds 


□  Delay  subsetial  end  by 
I  ~|  seconds 

Wind  flag  is  displayed 


I  Only  if  the  wind  speed  is  not  zero 


Wind  flag  is  drawn  with  a 
T  ransparent  background 
Wind  flag  is  located  in  the 


Lower  left  corner 


3 


Sub  Serial  Runtime 
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^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


:  Project  Edit  View  Tools  Window  Help 

i  ✓  y  Zoom  100%  ^  IJl  I  ^  f 


Sub  Serial  Preview 


Target  Details 
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Target  Lifetime 


Target  Position 
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i  Project  Edit  View  Tools  Window  Help 
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OPNAVINST3591.1F 
I  Verl  Navy  Handgun  Qual  Co 
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I  Unlimited  Practice 
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=  0  M9ZERO 
-  0  Subserial 

0  FATSZERO 
I  Zero  15  Yard  Line 


I  Details*  ||  Lifetime*  j  Position*  |  Behaviors*  |  Scoring*  ||  Regions*  | 


This  tab  associates  a  target's  responses  (behaviors)  to  things  that  occur  (events). 


Event 

Type 

Behavior 

Select  the  event 


1  The  target's  exposure  is  over 

1  Add  1 

Select  the  behavior  that  is  activated  by  the  event  above. 

1  Update  1 

a 


0  This  behavior  is  a  default  behavior  for  the  given  event 
O  This  behavior  replaces  the  default  behavior  for  the  path  node  belo¥w. 

The  target  disappears  after  the  event  has  occurred 
[l  ~  times. 

The  target  disappears 

1 0.0  ~|  seconds  after  the  the  required  number  of  hits  has  been  met. 


Errors  and  Warnings  | 
Ready 


Target  Behaviors 


Target  Scoring 
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S0®l 

i  Project  ^it  View  Tools  Window  Help 

i  lii^  H  Zoom  100%  » ^  ..  1  Its 1  ©  (§)  ^  y 

1-Lane  USER  M9  Courseware  ) 

<1  >  X 

qO  OPNAVINST  3591.  IF 

+  Verl  Navy  Handgun  Qual  Co 
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+  0  Unlimited  Practice 
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-  0  M9ZERO 
Q  ^  Subserial 

0  FATSZERO 
Q  0  Zero  15  Yard  Line 


I  Details*  ||  Lifetime*  ||  Position*  ||  Behaviors*  ||Tcoring*  |  Regions*  | 


[  Errors  and  Warnings  | 

Ready 


Region  and  Spotter  Color  Inforrrration 


Selected  region  will  alternate  white,  actual  color,  black  and  actual  color. 


Region  Max  Hits  Sc...  Bo...  Color 


+ 


Score  for  hitting  this  region 

|l  ^ points  per  hit 

Spotter  color  in  this  region  is 

Red  [O  ^  Green  [o  C  [Blue  [O  Q  [ 


Maximum  hits  for  this  region 
O  |l  C  I  hits 
0  No  limit 

Bonus  score  for  hitting  this  region  at  least  once 
^  points 


Target  Regions 
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APPENDIX  D.  15-YARD  ZERO  SCENARIO 


The  contents  of  the  appendix  are  screen  captures  of  the  15 -yard  zero  scenario. 
These  pictures  are  provided  so  that  anyone  can  recreate  the  course  of  fire  used  in  this 
experiment.  The  scenario  was  established  with  respect  to  Navy  Handgun  Qualification 
Course  found  in  the  OPNAV  INSTRUCTION  3591. IF  Small  Arms  Training  and 
Qualification. 
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Q  0  M9ZER015 
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0  FATSZERO 


Course  Name*  |  Course  Standards*  | 

Enter  the  name  of  this  course; 


Boresight  15  Yard  Line 

Practice  Names 

Number  Author  Name 

Display  Name 

"l  M9ZER015 

1 

< 

>1 

Enter  the  new  practice  Display  Name 

1  1  Update  1 

<  >1! _ ^ 

l|  Errors  and  Warnings  | 

1 Ready 

NUM  SCRL 

Course  Name 

thanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 
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:  Project  Wit  View  Tools  Window  Help 
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Q  0 
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0  M9ZER015 
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I  Course  Name*  |  Course  Standards*  | 


This  tab  defines  criteria  required  to  pass  this  course. 

Primary  course  standard  to  use 


Secondary  course  standard  to  use 

fOr  v]  I  None  ^v]| 


<  ^  >  _  _ 

^^rorsan^ 

Ready  NUM  SCRL 


Course  Standards 
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Type  I  Weapon  Ammunition*  |[  Preparation*  |  Runtime*  ||  Group  Assignment*  ||  Shot  Standards  1|  Standards  ||  Weapon  Sensor  Restrictions  ||  Scoring  |  Results*  | 

0iThis  practice  is  a  normal  practice.: 

O  T  nis  practice  uses  split  views 
Number  ot  split  view; 

O  This  practice  will  only  show  the  following  instructions: 

Language: 

I  English 


I  Errors  and  Warnings  | 

Ready  NUM  SCRL 


Practice  Type 


Practice  Weapon 
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S  ^  Unlimited  Practice 
+  ^  Ver2  Navy  Handgun  Qual  Co 
+  ^  Ver2S  Navy  Handgun  Qual  C 
~)  Zero  3  Yard  Line 
)  Zero  7  Yard  Line 
_)  Zero  15  Yard  Line 
-  0  Boresight  15  Yard  Line 
-  0  M9ZER015 
-  ^  Subserial 

0  FATSZERO 


I  Type  II  Weapon  ifArnmunition*  j|  Preparation*  |[  Runtime*  |  Group  Assignment*  ||  Shot  Standards  ||  Standards  1|  Weapon  Sensor  Restrictions  ||  Scoring  ||  Results*  | 

This  tab  defines  this  practice's  ammunition  quantity  and  characteristics. 

Ammunition  load; 


■ll 


1  Add  1 

1  Update  1 

1  Remove  | 

1  Move  Up  1 

|Move  Down| 

*  [  magazine(s)/belt(s)  containing 

*  I  lound(s)  per  magazine/belt. 


T  otal  rounds  added  by  this  practice. 

I  I  Override  the  default  weapon  ammunition  for  this  practice. 
Ammunition  type 

Define  Ammunition  on  a  Per  Round  Basis 
0  Use  tracer  effects. 

I  Every  round 

Define  Tracers  on  a  Per  Round  Basis 


:e  in  this  course  before  using  the 


0  Limit  the  number  of  shots 

f  lumber  of  allowed  shots  |l  0~| 

0  Use  sighters  at  the  beginning  of  the  practice. 
Number  of  sighters  | '  ] 


I  Errors  and  Warnings  | 

Ready 


Practice  Ammunition 


^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

BPSI 

:  Project  Edit  View  Tools  Window  Help 

i  Hi^  V*  Q  1  Jl  i3j  i  r.  -- 1  jf  fj 

Xl-Lane  USER  Mg  Courseware  | 

<1  >  X 

OPNAVINST3591.1F 
+  ^  Verl  Navy  Handgun  Qual  Co 
+  ^  VerlS  Navy  Handgun  Qual  C 
+  ^  Unlimited  Practice 
+  0  Ver2  Navy  Handgun  Qual  Co 
+  #  Ver2S  Navy  Handgun  Qual  C 
S  ^  Zero  3  Yard  Line 
S  0  Zero  7  Yard  Line 
B  ^  Zero  15  Yard  Line 

-  0  Boresight  15  Yard  Line 
-  0  M9ZER015 
-  0  Subserial 

0  FATSZERO 


I  Type  II  Weapon  ||  Ammunition*  |  Preparation*  |  Runtime*  ||  Group  Assignment*  ||  Shot  Standards  1|  Standards  ||  Weapon  Sensor  Restrictions  ||  Scoring  ||  Results*  | 


I  Errors  and  Warnings  | 

Ready 


Enter  the  first  page  of  instructions  if  desired 

Language:  _ 

[^English 
I  No  Time  Limit 


Enter  the  second  page  of  instructions  if  desired 


[l  Mag 


1 3  Rounds" 


Enter  the  text  for  the  Load  Weapon  prompt  if  desired 

Wait  For  Instructions _ | 

0|Show  targets  at  their  initial  position  during  preview 


Practice  Preparation 
104 


Practice  Runtime 


Practice  Group  Assignment 
105 


I  Errors  and  Warnings 
Ready 


Practice  Shot  Standards 


^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

i  Project  Wit  Wew  Tools  Window  Help 

iE^ ✓  y  J  ^ 

1-Lane  USER  M9  Courseware  | 

<1  >  X 

1 

B  O  OPNAVINST3591.1F 

a  (§)  Verl  Navy  Handgun  Qual  Co 
a  (§)  VerlS  Navy  Handgun  Qual  C 
a  (f)  Unlimited  Practice 
a  d)  Ver2  Navy  Handgun  Qual  Co 
a  d)  Ver2S  Navy  Handgun  Qual  C 
a  d)  2ero  3  Yard  Line 
a  d)  2ero  7  Yard  Line 
a  d)  2ero  15  Yard  Line 

-  0  Boresight  15  Yard  Line 
B  0  M9ZER015 
a  0  Subserial 

0  FATS2ERO 


^ _  > 

[  Errors  and  Warnings  | 

Ready  NUM  SCRL 


I  B  Weapon  |  Ammunition*  |  Preparation*  |  Runtime*  |  Group  Assignment*  ||  Shot  Standards  |  Standards  |  Weapon  Sensor  Restrictions  ||  Scoring  ||  Results*  | 


This  tab  is  used  to  define  criteria  required  to  pass  this  practice.  Please  configure  the  standards  after  you 
have  created  all  subserials  and  targets  within  the  practice. 

Primary  practice  standard  to  use 


Each  target  must  be  hit  at  least  one  time  to  pas-s  this  practice 


I  None 


I 


Practice  Standards 
106 


Practice  Weapon  Sensor  Restrietions 


Practice  Scoring 
107 


Practice  Results  Top 


Practice  Results  Bottom 
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Sub  Serial  Preparation 


Sub  Serial  Scene 
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Sub  Serial  Ballistic  Terrain  Lines 


^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

Bll®l 

i  Project  Wit  View  Tools  Window  Help 

i  Hi^  ^  Q  Zoom  100%  •r  ^  “  y 

X  1-Lane  USER  M9  Courseware  ] 

0  >  X 

0  OPNAVINST3591.1F 
+  ^  Verl  Navy  Handgun  Qual  Co 
S  %  VerlS  Navy  Handgun  Qual  C 
+  ^  Unlimited  Practice 
13  ^  Ver2  Navy  Handgun  Qual  Co 
S  ^  Ver2S  Navy  Handgun  Qual  C 
13  4^  Zero  3  Yard  Line 
13  ^  Zero  7  Yard  Line 
B  ^  Zero  15  Yard  Line 

-  0  Boresight  15  Yard  Line 
B  0  M9ZER015 
=1  0  Subserial 

0  FATSZERO 


I  Preparation*]!  Scene*  ||  Ballistic  Terrain  Lines*  |  Runtime*  |  Previet^ 

This  tab  is  used  to  set  options  used  during  the  tunning  of  this  subsetial. 

Subseiial  runs  until 


il  practice  is  fired  or  all  targets  have  timed  out. 


I  Errors  and  Warnings  | 

Ready 


|l  t]  shots 

□  Delay  the  display  of  shot  results  in  the  immediate  feedback  area. 

[Z  Q  I  second  : 

□  Delay  subsetial  end  by 

I  ~|  seconds 

Wind  flag  is  displayed 


1  Only  if  the  wind  speed  is  not  zero 

Wind  flag  is  drawn  with  a 

1  T  ransparent  background 

Wind  flag  is  located  in  the 

1  Lower  left  cornet 

-1 

Sub  Serial  Runtime 
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afgBi 


^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


Project  yit  View  Tools  Window  Help 
S’  y  Q  Zoom  100%  ^  % 


1-Lane  USER  M9  Courseware 


OPNAVINST  3591.  IF 

Verl  Navy  Handgun  Qual  Co 
VerlS  Navy  Handgun  Qual  C 
Unlimited  Practice 
Ver2  Navy  Handgun  Qual  Co 
Ver2S  Navy  Handgun  Qual  C 
Zero  3  Yard  Line 
Zero  7  Yard  Line 
Zero  15  Yard  Line 

Boresight  15  Yard  Line 
M9ZER015 
Subserial 
FATSZERO 


Preparation*  ||  Scene*  [|  Ballistic  Terrain  Lines*  ||  Runtime*  |  Preview*  | 


Advance]  |l  Oj  seconds  O  Pause  if  an  event  occurs  Elapsed  lime:  |0.0  |  Gun:|l  y|  Shots:  |0  | 


[  Errors  and  Warnings  ] 

Ready  NUM  SCRL 


Sub  Serial  Preview 


Target  Details 
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Target  Lifetime 


Target  Position 
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^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

:  Project  ^it  View  Tools  Window  Help 

i  lii^:  Q  Zoom  100%  Ife  0  <§)  ^ 

,/  1-Lane  USER  M9  Courseware  ] 

<1  >  X 

Q  ©  OPNAVINST3591.1F 

+  ^  Verl  Navy  Handgun  Qual  Co 
S  ^  VerlS  Navy  Handgun  Qual  C 
S  ^  Unlimited  Practice 
+  ^  Ver2  Navy  Handgun  Qual  Co 
+  ^  Ver2S  Navy  Handgun  Qual  C 
Zero  3  Yard  Line 
Zero  7  Yard  Line 
_  Zero  15  Yard  Line 
-  0  Boresight  15  Yard  Line 
-  0  M9ZER015 
T  ^  Subserial 

0  FATSZERO 


I  Details*  ||  Lifetime*  ||  Position*  |  Behaviors*  |  Scoring*  ||  Regions*  | 


■ll 


a  target's  responses  (behaviors)  to  things  that  occur  (events). 


Event  Type  Behavior 

< 

> 

Select  the  event 

1  The  target's  exposure  is  over 

vll  (  Add  1 

Select  the  behavior  that  is  activated  by  the  event  above. 

1  Update  1 

I  Remove  the  target 


0  This  behavior  is  a  default  behavior  for  the  given  event 
O  This  behavior  replaces  the  default  behavior  for  the  path  node  below. 

The  target  disappears  after  the  event  has  occurred 
|l  ^  times. 

The  target  disappears 

1 0.0  ~|  seconds  after  the  the  required  number  of  hits  has  been  met. 


Errors  and  Warnings  | 

Ready 


Target  Behaviors 


Target  Scoring 
113 


Target  Regions 
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APPENDIX  E.  UNLIMITED  PRACTICE  SCENARIO 


The  contents  of  the  appendix  are  screen  captures  of  the  unlimited  practice 
scenario.  These  pictures  are  provided  so  that  anyone  can  recreate  the  course  of  fire  used 
in  this  experiment.  The  scenario  was  established  with  respect  to  Navy  Handgun 
Qualification  Course  found  in  the  OPNAV  INSTRUCTION  3591. IF  Small  Arms 
Training  and  Qualification. 


Sub  Stage 
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BfgfKII 


^^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


;  Project  Edit  View  Tools  Window  Help 


1-Lane  USER  M9  Courseware  | 

^  OPNAVINST3591.1F 
f  ^  Verl  Navy  Handgun  Qual  Co 
f  #  VerlS  Navy  Handgun  Qual  C 
-  ^  Unlimited  Practice 

-  0QKB 

UNLIMITED 
-  O  Subserial 
©  Trans2 
©  Trans2 
©  Trans2 

*•  ^  Ver2  Navy  Handgun  Qual  Co 
^  ^  Ver2S  Navy  Handgun  Qual  C 
^  %  Zero  3  Yard  Line 
^  ^  Zero  7  Yard  Line 
^  ^  Zero  15  Yard  Line 


[  Errors  and  Warnings  | 

Ready 


;e  Name*  |  course  Standards*  | 


Enter  the  name  of  this  course; 


1  Unlimiled  Course  of  Fire 

1 

Practice  Names 

Number  Author  Name 

Display  Name 

1  Reset  1 

“l  UNLIMITED 

1 

1  >: 

Enter  the  new  practice  Display  Name 

1 

1 1  Update  1 

Course  Name 


^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

i  Project  Edit  View  Tools  Window  Help 

isSVH  1  1  ^ 

1 -Lane  USER  M9  Courseware  ] 

<1  >  X 

;>  OPNAVINST3591.1F 
H  0  Verl  Navy  Handgun  Qual  Co 
r  ^  VerlS  Navy  Handgun  Qual  C 
3  ^  Unlimited  Practice 

©  Unlimited  Course  of  Fire 
a  0  UNLIMITED 
a  ©  Subserial 
©  Trans2 
“>  Trans2 
J'  Trans2 
t)  ©  Ver2  Navy  Handgun  Qual  Co 
^  ©  Ver2S  Navy  Handgun  Qual  C 
I-  ©  Zero  3  Yard  Line 
r  ©  Zero  7  Yard  Line 
r  ©  Zero  15  Yard  Line 


I  Course  Name*  |  Course  Standards*  | 


©T 


This  tab  defines  criteria  required  to  pass  this  course. 

Primary  course  standard  to  use 


Secondary  course  standard  to  use 
[Or ^  I  None  ~ 


Errors  and  Warnings  | 

Ready 


Course  Standards 
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I^C^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


i  Project  Edit  View  Tools  Window  Help 


i-Lane  USER  M9  Coursewai^ 


OPNAVINST3591.1F 
+  ^  Verl  Navy  Handgun  Qual  Co 
■f  #  VerlS  Navy  Handgun  Qual  C 
-  ^  Unlimited  Practice 

^  Unlimited  Course  oF  Fire 

(BH - 

Subserial 
Trans2 
^  Trans2 
^  Trans2 

\  Ver2  Navy  Handgun  Qual  Co 
\  Ver2S  Navy  Handgun  Qual  C 
\  Zero  3  Yard  Line 
\  Zero  7  Yard  Line 
\  Zero  15  Yard  Line 


Type  I  Weapon  ||  Ammunition*  ||  Preparation*  ||  Runtime*  ||  Group  Assignment*  ||  Shot  Standards  ||  Standards  ||  Weapon  Sensor  Restrictions  ||  Scoring  ||  Results*  | 

0  This  practice  is  a  normal  practice. 

O  This  practice  uses  split  views 
Number  of  split  views 

O  This  practice  will  only  show  the  following  instructions: 

Language: 


<  I  >  I 

I  Errors  and  Warnings  | 

Ready  NUM  SCRL 


Practice  Type 


Practiee  Weapon 
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l^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


[  Errors  and  Warnings 

Ready 


Practice  Preparation 


118 


Practice  Runtime 


Practice  Group  Assignment 


119 


Practice  Shot  Standards 


Practice  Standards 


120 


Practice  Weapon  Sensor  Restrietions 


Practice  Scoring 
121 


Practice  Results  Top  Half 


Practice  Results  Bottom  Half 


122 


O^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

□(1® 

:  Project  Edit  View  Tools  Window  Help 

X  1-Lane  USER  M9  Courseware  ] 

<1  >  X 

B  0  OPNAVINST3591.1F 

11  (§)  Verl  Navy  Handgun  Qual  Co 
11  d)  VerlS  Navy  Handgun  Qual  C 
B  (§)  Unlimited  Practice 

B  0  Unlimited  Course  of  Fire 
B  0  UNLIMITED 

B  0IEEESS1 

0  Trans2 
0  Trans2 
0  Trans2 

+  0  Ver2  Navy  Handgun  Qual  Co 
II  0  Ver2S  Navy  Handgun  Qual  C 
B  0  Zero  3  Yard  Line 
II  0  Zero  7  Yard  Line 
Zero  15  Yard  Line 


Preparation*  |  Scene*  [[  Ballistic  Terrain  Lines*  |  Runtime^]  Preview*  | _ 

This  tab  allows  you  to  pause  and  optionally  specify  instuctions  before  each  subserial  starts. 

[~~l  Pause  before  subseiial  starts  (not  usable  for  the  first  subserial  in  a  practice) 


Language: 


Errors  and  Warnings  1 


Sub  Serial  Preparation 


Sub  Serial  Scene 
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Errors  and  Warnings  j 

Ready  NUM  SCRL 


Sub  Serial  Ballistic  Terrain  Lines 


Sub  Serial  Runtime 
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Sub  Serial  Preview 


3-Yard  Target  Details 
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3 -Yard  Target  Lifetime 


3 -Yard  Target  Position 
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l^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


3 -Yard  Target  Behaviors 


planes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

i  Project  Edit  View  Tools  Window  Help 

i  ✓  H  Zoom  1007o  ^  T.  . 

/ 1 -Lane  USER  M9  Courseware  | 

<1  P  X 

0  OPNAVINST3591.1F 
IS  (§)  Verl  Navy  Handgun  Qual  Co 
IS  (§)  VerlS  Navy  Handgun  Qual  C 
S  (§)  Unlimited  Practice 

S  O  Unlimited  Course  of  Fire 
S  ®  UNLIMITED 
1=1  0  Subserial 
0  Trans2 
0  Trans2 
0  Trans2 

S  (§)  Ver2  Navy  Handgun  Qual  Co 
S  (§)  Ver2S  Navy  Handgun  Qual  C 
S  d)  Zero  3  Yard  Line 
IS  d)  Zero  7  Yard  Line 
IS  d)  Zero  15  Yard  Line 


I  Details*  ||  Lifetime*  ||  Position*  ||  Behaviors*  |  Scoring*  |  Regions*  | 

This  tab  defines  how  the  target  is  scored. 

The  target  is 


I  I  This  target  can  only  be  hit  by  a  specific  weapon 


□  Limit  the  number  of  times  that  the  target  can  be  hit  to 


When  scoring  by  group  size,  the  following  group  sizes  are  worth  the  following  points. 

Group  sizes  less  than  or  equal  to 


Group  Size  Score 

1  Add  1 

1  Update  1 

1  Remove  | 

1 5.0  |cr 
are  worth 

|5  point(s). 


< _ lJ==^=^^==^^=^^=====^================^^=! 

[  Errors  and  Warnings  | 

Ready  NUM  SCRL 


3 -Yard  Target  Scoring 
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^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 


:  Project  ^it  View  Tools  Window  Help 
i  V  Q  Zoom  100%  -  JL  ife  ' 


0  <§)  ^ 


^  1-Lane  USER  M9  Courseware 


a  0  OPNAVINST3591.1F 

+  d)  Verl  Navy  Handgun  Qual  Co 
S  (§)  VerlS  Navy  Handgun  Qual  C 
Q  d)  Unlimited  Practice 

B  O  Unlimited  Course  of  Fire 
B  0  UNLIMITED 
-  0  Subserial 
0  Trans2 
0  Trans2 
0'  Trans2 

+  0  Ver2  Navy  Handgun  Qual  Co 
+  0  Ver2S  Navy  Handgun  Qual  C 
B  0  Zero  3  Yard  Line 
B  0  Zero  7  Yard  Line 
S  0  Zero  15  Yard  Line 


I  Details*  ||~Lifetime*  Ifposition*  ll'oehaviors*  ||  Scoring*  |  Regions*  | 


Region  and  Spotter  Color  Information 


Selected  region  will  alternate  white,  actual  color,  black  and  actual  color. 


Region 

Max  Hits 

Sc... 

Bo... 

Color 

3 

No  Limit 

3 

0 

0,0,0 

4 

No  Limit 

4 

0 

0,0,0  J 

5 

No  Limit 

5 

0 

0,0,0 

6 

No  Limit 

5 

0 

0,0,0 

6 


Score  lor  hitting  this  region 


I  points  p 


Spotter  color  in  this  region  is 

Red  [o _ ^  Green  [o  y[Blue  |o  Q  | 


Maximum  hits  for  this  region 

oE 


0  No  lir 

Bonus  score  for  hitting  this  region  at  least  once 
,^1  points 


I  Errors  and  Warnings  | 

Ready 


3 -Yard  Target  Regions 


7- Yard  Target  Details 


128 


7-Yard  Target  Lifetime 


7- Yard  Target  Position 
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C^Lanes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

BH® 

;  Project  ^it  View  Tools  Window  Help 

I  S#  ✓  H  Zoom  100%  ^  (fe  Ti  0  0  ^  y 

y/  1-Lane  USER  M9  Courseware  | 

<1  >  X 

s  O 


a  (D 


OPNAVINST  3591.  IF 

Verl  Navy  Handgun  Qual  Co 
VerlS  Navy  Handgun  Qual  C 
_  Unlimited  Practice 
a0  Unlimited  Course  of  Fire 
a  0  UNLIMITED 
a  0  Subserial 
O  Trans2 
0  Trans2 
0  Trans2 

0  Ver2  Navy  Handgun  Qual  Co 
0  Ver2S  Navy  Handgun  Qual  C 
0  Zero  3  Yard  Line 
0  Zero  7  Yard  Line 
0  Zero  15  Yard  Line 


[  Errors  and  Warnings  ] 

Ready 


Details*  |  Lifetime*  ||^sition*  |  Behaviors*  |  Scoring*  1|  Regions*  | 


This  tab  associates  a  target's  responses  (behaviors)  to  things  that  occur  (events). 


Event 

Type 

Behavior 

The  target's  exposure  r 

< 

s  over  Default 

Disappear  after  the  event  occurs  1  time  (s)  using  a  delay  of  0.00  seconds. 

> 

Select  the  event 


1  The  target  is  hit 

[  Add  1 

Select  the  behavior  that  is  activated  by  the  event  above. 

1  Update  1 

0  This  behavior  is  a  default  behavior  lor  the  given  event 
O  This  behavior  replaces  the  default  behavior  for  the  path  node  below. 


d 


7- Yard  Target  Behaviors 


7- Yard  Target  Scoring 
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7-Yard  Target  Regions 


^^anes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

:  Project  Edit  View  Tools  Window  Help 

I  ^  Q  Zoom  100%  -JJjL  -  ©  0  ^  ^  0 

1-Lane  USER  M9  Courseware  | 

<1  >  X 

OPNAVINST3591.1F 
IS  ^  Verl  Navy  Handgun  Qual  Co 
S  ^  VerlS  Navy  Handgun  Qual  C 
1$  ^  Unlimited  Practice 

-  ^  Unlimited  Course  of  Fire 
Id  0  UNLIMITED 
Q  ^  Subserial 
O  Trans2 
0  Trans2 
0  Trans2 

I-  ^  Ver2  Navy  Handgun  Qual  Co 
h  0  Ver2S  Navy  Handgun  Qual  C 
B  ^  Zero  3  Yard  Line 
B  ^  Zero  7  Yard  Line 
B  ^  Zero  15  Yard  Line 


n  Lifetime*  ||  Position*  ||  Behaviors*  ||  Scoring*  ||  Regions*  | 


Point  Type  Location 

[  Add  New  Point  |  |  CZP  v  | 

T  argel 

1  Default  CZP  (30,50) 

1  Override  Point  |  X  |30  | 

pTransZ.bmp  vj 

Y  |9)  1 

Animation  speed 

1  ~|  frames  per  second 

1  Undo  Override  | 

^o-t> 


I  Errors  and  Warnings  | 

Ready 


15-Yard  Target  Details 
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15- Yard  Target  Lifetime 


15- Yard  Target  Position 
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[planes  Authoring  (USER)  -  [1  -Lane  USER  M9  Courseware] 

fSTlfxl 

i  Project  Edit  View  Tools  Window  Help 

I  V*  y  Zoom  100%  ^  1^  0  <§>  ^  ini  S 

^1-Lane  USER  M9  Courseware  | 

<  t>  X 

a  o  OPNAVINST  3591 .  IF  I  Details*  ||  Lifetime*  ||  Position*  I  Behaviors*  I  Scoring*  ||  Regions*  I 

SI  (§)  Verl  Navy  Handgun  Qual  Co 
Q  (§)  VerlS  Navy  Handgun  Qual  C 
B  (§)  Unlimited  Practice 

Q  ^  Unlimited  Course  of  Fire 
a  ®  UNLIMITED 
Id  ^  Subserial 
O  Trans2 
0  Trans2 
0:'  Trans2 

IS  (§)  Ver2  Navy  Handgun  Qual  Co 
IS  (§)  Ver2S  Navy  Handgun  Qual  C 
IS  (§)  Zero  3  Yard  Line 
IS  (§)  Zero  7  Yard  Line 
IS  d)  Zero  15  Yard  Line 


< _ ^ 

[  Errors  and  Warnings  | 

Ready  NUM  SCRL 


15- Yard  Target  Behaviors 


15-Yard  Target  Scoring 
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15- Yard  Target  Regions 
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APPENDIX  F.  PRACTICE  QUALIFICATION  SCENARIO 


The  contents  of  the  appendix  are  screen  captures  of  the  practice  qualification 
scenario.  These  pictures  are  provided  so  that  anyone  can  recreate  the  course  of  fire  used 
in  this  experiment.  The  scenario  was  established  with  respect  to  Navy  Handgun 
Qualification  Course  found  in  the  OPNAV  INSTRUCTION  3591. IF  Small  Arms 
Training  and  Qualification. 
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Event 
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Select  the  event 

1  The  target's  exposure  is  over 

[  Add  1 

Select  the  behavior  that  is  activated  by  the  event  above. 

1  Update  1 
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O  This  behavior  replaces  the  default  behavior  for  the  path  node  below. 


The  target  disappears  after  the  event  has  occurred 

h  ^ 
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[o.O  ~[  seconds  after  the  the  required  number  of  hits  has  been  met. 
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APPENDIX  G.  DEMOGRAPHIC  SURVEY 


A  Comparison  of  Current  Naval  Marksmanship  Training  VS 
Simulation  Based  Marksmanship  Training  with  the  use  of  Indoor 
Simulation  Marksmanship  Trainer  (ISMT) 

Purpose:  This  research  is  intended  to  provide  naval  personnel  with  the  option  to  using 
simulation  for  the  purpose  of  basic  training  necessary  to  meet  a  qualification  standard.  The 
purpose  for  doing  this  research  is  to  provide  a  more  robust  method  for  conducting 
marksmanship  training  in  the  Navy. 

# _ Note:  Please  choose  a  four  digit  number  for  data  recording  purposes.  Do  not  use 

the  last  four  of  your  social.  This  information  will  keep  your  identity  anonymous 

Demographic  questions: 

Directions:  Answer  the  following  questions  or  circle  the  answer  that  applies  to  you. 

1.  Are  you  (circle  one  age  group):  18-25  26-30 

31-35  36-40  41+ 

2.  What  is  your  height?(ex.  5'  8'') _ 

3.  What  is  your  weight?  (180  lbs) _ 

4.  Are  you  a  (circle  one):  Male  Female 

5.  Are  you  (circle  one):  Right  Handed  Left  Handed  Both/Ambidextrous 

6.  Do  you  wear  corrective  lenses,  ex.  glasses  or  contacts  (circle  one):  Yes  No 

7.  Are  you  color  deficient?  (circle  one):  Yes  No 

8.  Are  you  (circle  one):  Active  duty  Reserves  Veteran  Civilian 

a.  If  you  are  Active  duty,  which  branch  of  service  (circle  one): 

Navy  Marine  Corp  Airforce  Army  International  Student 

b.  What  is  your  Rank?  (ex.  LT) _ 

c.  Years  of  Service?  (ex.  6yrs) _ 

d.  What  is  your  specialty  (ex.  SWO,  Pilot,  etc.)? 


9.  Have  you  ever  fired  a  gun?  (circle  one):  Yes  No 

a.  If  you  answered  yes,  describe  what  kind: _ 

10.  How  would  you  rate  your  level  as  a  marksman  right  now?  (circle  one): 

i.  Novice 

ii.  Marksman 

iii.  Sharpshooter 

iv.  Expert 

11.  Have  you  ever  received  any  formal  marksmanship  training?  (circle  one):  Yes  No 

a.  When  was  your  last  training?  (MMM/YY) _ 

12.  Have  you  ever  fired  a  gun  on  a  scored  course  of  fire?  (circle  one):  Yes  No 

a.  If  you  answered  yes,  when  was  the  last  time:  (MMM/YY) _ 

b.  What  type  of  gun  was  used?  (ex.  9M  qual,  M-16  qual) _ 

c.  What  was  your  performance  level/score?  (ex.  Expert,  or  240) _ 

13.  Have  you  ever  used  a  Marksmanship  simulator?  (circle  one):  Yes  No 
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APPENDIX  H.  POST  SURVEY 


A  Comparison  of  Current  Naval  Marksmanship  Training  VS 
Simulation  Based  Marksmanship  Training  with  the  use  of  Indoor 
Simulation  Marksmanship  Trainer  (ISMT) 

Purpose:  This  research  is  intended  to  provide  naval  personnel  with  the  option  to  using 
simulation  for  the  purpose  of  basic  training  necessary  to  meet  a  qualification  standard.  The 
purpose  for  doing  this  research  is  to  provide  a  more  robust  method  for  conducting 
marksmanship  training  in  the  Navy. 

# _ Note:  Please  choose  a  four  digit  number  for  data  recording  purposes.  Do  not  use 

the  last  four  of  your  social.  This  information  will  keep  your  identity  anonymous 

Post  Critique 

Directions:  Write  a  short  answer  for  the  following  questions.  Please  print  clearly. 

1.  How  useful  do  you  think  simulators,  like  the  Indoor  Simulated  Marksmanship  Trainer,  would 
be  for  marksmanship  training? 


2.  Do  you  think  simulated  training  will  ever  replace  ''real''  training? 


3.  Explain  your  experience  with  using  ANY  simulations  for  formal  training?  (ex.  Ship  handling, 
simulator,  flight  simulator,  motorcycle  simulator) 


4.  Any  additional  comments. 
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